2024-04-17T07:31:23.309 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T07:31:23.309 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} --archive /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912 -- /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/orig.config.yaml 2024-04-17T07:31:23.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T07:31:23.456 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912 branch: wip-rishabh-testing-20240416.193735 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7659912' kernel: client: branch: testing flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a8c84f5997950fd7d008204839253291d52e6eda subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3521 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 410/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI8YCbSQKJnZDWMiRzOO/nNlJCnrRrKWAdZnsZ32+4q5PTZL7pN1fyrKpFrN5JA8nfZTtBGAOvydLOupfZsAYc4= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWzDvJkyYvkxzcFgX+K/t5Qfyb7gpPrXpt0dSdzfRIF/g6onXa7y0plhTMuaHKUQ9mGzgXdtUprrzPnWvn0Cxg= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHUEHVICN3K3rqyRYx+RBRTycjx+SxXCcH+d9/LUJM9/Ppo+I0EmyvFZIMRwVwsh+OFC0XT8TKvE5uIjU+ko6fk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-17_04:58:44 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T07:31:23.456 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-17T07:31:23.457 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-17T07:31:23.457 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T07:31:23.458 INFO:teuthology.task.internal:Checking packages... 2024-04-17T07:31:23.480 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-17T07:31:23.481 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T07:31:23.481 INFO:teuthology.packaging:ref: None 2024-04-17T07:31:23.481 INFO:teuthology.packaging:tag: None 2024-04-17T07:31:23.481 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-17T07:31:23.481 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:31:23.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-17T07:31:23.696 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-17T07:31:23.698 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T07:31:23.704 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T07:31:23.704 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T07:31:23.710 INFO:teuthology.task.internal:Saving configuration 2024-04-17T07:31:23.730 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T07:31:23.736 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T07:31:23.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 07:26:14.683903', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI8YCbSQKJnZDWMiRzOO/nNlJCnrRrKWAdZnsZ32+4q5PTZL7pN1fyrKpFrN5JA8nfZTtBGAOvydLOupfZsAYc4='} 2024-04-17T07:31:23.783 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 07:26:14.682253', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWzDvJkyYvkxzcFgX+K/t5Qfyb7gpPrXpt0dSdzfRIF/g6onXa7y0plhTMuaHKUQ9mGzgXdtUprrzPnWvn0Cxg='} 2024-04-17T07:31:23.803 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 07:26:14.680369', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHUEHVICN3K3rqyRYx+RBRTycjx+SxXCcH+d9/LUJM9/Ppo+I0EmyvFZIMRwVwsh+OFC0XT8TKvE5uIjU+ko6fk='} 2024-04-17T07:31:23.803 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T07:31:23.809 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-17T07:31:23.809 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-17T07:31:23.809 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-17T07:31:23.809 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T07:31:23.906 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff03acc4160>, signals=[15]) 2024-04-17T07:31:23.907 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T07:31:23.912 INFO:teuthology.task.internal:Opening connections... 2024-04-17T07:31:23.913 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-04-17T07:31:23.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:31:24.019 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-04-17T07:31:24.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:31:24.125 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-04-17T07:31:24.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:31:24.229 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T07:31:24.236 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-04-17T07:31:24.254 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-04-17T07:31:24.254 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-04-17T07:31:24.310 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T07:31:24.311 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T07:31:24.312 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-04-17T07:31:24.334 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-04-17T07:31:24.352 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-04-17T07:31:24.353 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-04-17T07:31:24.410 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-04-17T07:31:24.410 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-04-17T07:31:24.410 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-04-17T07:31:24.410 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T07:31:24.411 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T07:31:24.411 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-04-17T07:31:24.433 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-04-17T07:31:24.451 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-04-17T07:31:24.451 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-04-17T07:31:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-04-17T07:31:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-04-17T07:31:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-04-17T07:31:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-04-17T07:31:24.508 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T07:31:24.509 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T07:31:24.509 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-04-17T07:31:24.525 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T07:31:24.535 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T07:31:24.540 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T07:31:24.540 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-04-17T07:31:24.543 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-04-17T07:31:24.545 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-04-17T07:31:24.564 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T07:31:24.569 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T07:31:24.570 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-04-17T07:31:24.600 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-04-17T07:31:24.603 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-04-17T07:31:24.635 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T07:31:24.709 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T07:31:24.723 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-17T07:31:24.724 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-17T07:31:24.724 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-17T07:31:24.724 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T07:31:24.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:31:24.741 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-17T07:31:24.980 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T07:31:24.981 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-17T07:31:24.981 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-04-17T07:31:24.998 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-361.el9.x86_64 2024-04-17T07:31:24.999 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T07:31:24.999 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-17T07:31:25.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum upgrade -y linux-firmware 2024-04-17T07:31:27.263 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-17T07:31:28.781 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2024-04-17T07:31:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-04-17T07:31:54.328 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 882 kB/s | 19 MB 00:22 2024-04-17T07:31:58.728 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-04-17T07:32:00.212 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-17T07:32:01.164 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-17T07:32:06.389 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-17T07:32:08.988 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-04-17T07:32:08.999 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-04-17T07:32:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-04-17T07:32:08.999 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 1 Package 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 375 M 2024-04-17T07:32:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-04-17T07:32:15.898 INFO:teuthology.orchestra.run.smithi046.stdout:linux-firmware-20240219-143.el9.noarch.rpm 54 MB/s | 375 MB 00:06 2024-04-17T07:32:15.899 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-04-17T07:32:15.900 INFO:teuthology.orchestra.run.smithi046.stdout:Total 54 MB/s | 375 MB 00:06 2024-04-17T07:32:15.900 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-04-17T07:32:15.985 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-04-17T07:32:15.985 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-04-17T07:32:18.473 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-04-17T07:32:18.473 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-04-17T07:32:24.931 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-04-17T07:32:25.144 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T07:32:25.242 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T07:32:26.604 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T07:32:26.605 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T07:32:26.993 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T07:32:26.993 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:32:26.993 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-04-17T07:32:26.993 INFO:teuthology.orchestra.run.smithi046.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-17T07:32:26.993 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:32:26.993 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-04-17T07:32:27.309 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-17T07:32:27.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T07:32:27.454 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-17T07:32:27.454 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-17T07:32:27.457 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-04-17T07:32:39.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17 07:32:39 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-17T07:32:39.829 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2024-04-17 07:32:39-- 2024-04-17T07:32:39.829 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 12s 2024-04-17T07:32:39.829 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 341M in 12s (28.5 MB/s) 2024-04-17T07:32:39.832 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-17T07:32:39.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-17T07:32:41.985 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying... ######################################## 2024-04-17T07:32:42.044 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing... ######################################## 2024-04-17T07:32:42.128 INFO:teuthology.orchestra.run.smithi046.stdout:Updating / installing... 2024-04-17T07:32:50.417 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-17T07:33:07.123 INFO:teuthology.orchestra.run.smithi046.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-17T07:33:30.851 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-17T07:33:30.852 DEBUG:teuthology.orchestra.run.smithi046:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-17T07:33:30.936 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:30.936 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:30.936 INFO:teuthology.orchestra.run.smithi046.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:30.936 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-17T07:33:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-17T07:33:30.938 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-17T07:33:30.939 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-17T07:33:30.940 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-17T07:33:30.941 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-17T07:33:30.942 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-17T07:33:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-17T07:33:30.944 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-17T07:33:30.945 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-17T07:33:30.946 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-17T07:33:30.947 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-17T07:33:30.948 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-17T07:33:30.949 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-17T07:33:30.950 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-17T07:33:30.951 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-17T07:33:30.952 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-17T07:33:30.953 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-17T07:33:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-17T07:33:30.955 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-17T07:33:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-17T07:33:30.957 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-17T07:33:30.958 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-17T07:33:30.959 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-17T07:33:30.960 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-17T07:33:30.961 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-17T07:33:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-17T07:33:30.963 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-17T07:33:30.964 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-17T07:33:30.965 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-17T07:33:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-17T07:33:30.967 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-17T07:33:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-17T07:33:30.969 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-17T07:33:30.970 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-17T07:33:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-17T07:33:30.972 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-17T07:33:30.973 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-17T07:33:30.974 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-17T07:33:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-17T07:33:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-17T07:33:30.977 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-17T07:33:30.978 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-17T07:33:30.979 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-17T07:33:30.980 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-17T07:33:30.981 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-17T07:33:30.982 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-17T07:33:30.983 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-17T07:33:30.984 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-17T07:33:30.985 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-17T07:33:30.986 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-17T07:33:30.987 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-17T07:33:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-17T07:33:30.989 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-17T07:33:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-17T07:33:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-17T07:33:30.992 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-17T07:33:30.993 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-17T07:33:30.994 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-17T07:33:30.995 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-17T07:33:30.996 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-17T07:33:30.997 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-17T07:33:30.998 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-17T07:33:30.999 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-17T07:33:31.000 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-17T07:33:31.001 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-17T07:33:31.002 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-17T07:33:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-17T07:33:31.004 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-17T07:33:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-17T07:33:31.006 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-17T07:33:31.007 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-17T07:33:31.008 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-17T07:33:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-17T07:33:31.010 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-17T07:33:31.011 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-17T07:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-17T07:33:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-17T07:33:31.014 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-17T07:33:31.015 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-17T07:33:31.016 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-17T07:33:31.017 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-17T07:33:31.018 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-17T07:33:31.019 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-17T07:33:31.020 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-17T07:33:31.021 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-17T07:33:31.022 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-17T07:33:31.023 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-17T07:33:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-17T07:33:31.025 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-17T07:33:31.026 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-17T07:33:31.027 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-17T07:33:31.028 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-17T07:33:31.029 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-17T07:33:31.030 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-17T07:33:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-17T07:33:31.032 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-17T07:33:31.033 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-17T07:33:31.034 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-17T07:33:31.035 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-17T07:33:31.036 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-17T07:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-17T07:33:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-17T07:33:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-17T07:33:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-17T07:33:31.041 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-17T07:33:31.042 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-17T07:33:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-17T07:33:31.044 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-17T07:33:31.045 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-17T07:33:31.046 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-17T07:33:31.047 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-17T07:33:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-17T07:33:31.049 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-17T07:33:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-17T07:33:31.051 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-17T07:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-17T07:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-17T07:33:31.054 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-17T07:33:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-17T07:33:31.056 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-17T07:33:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-17T07:33:31.058 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-17T07:33:31.059 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-17T07:33:31.060 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-17T07:33:31.061 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-17T07:33:31.062 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-17T07:33:31.063 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-17T07:33:31.064 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-17T07:33:31.065 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-17T07:33:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-17T07:33:31.067 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-17T07:33:31.068 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-17T07:33:31.069 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-17T07:33:31.070 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-17T07:33:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-17T07:33:31.072 INFO:teuthology.orchestra.run.smithi046.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T07:33:31.076 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:31.077 DEBUG:teuthology.orchestra.run.smithi046:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout:postinstall scriptlet (using /bin/sh): 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout:for file in vmlinuz System.map config; do 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T07:33:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: fi 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:31.101 INFO:teuthology.orchestra.run.smithi046.stdout:fi 2024-04-17T07:33:31.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-04-17T07:33:31.188 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-04-17T07:33:31.188 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-04-17T07:33:31.188 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-04-17T07:33:31.188 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 68.el9 2024-04-17T07:33:31.188 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8264254 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-04-17T07:33:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:33:31.190 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T07:33:31.190 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T07:33:31.190 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T07:33:31.190 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-04-17T07:33:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:33:31.190 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T07:33:31.191 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T07:33:31.191 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:31.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T07:33:31.572 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-04-17T07:33:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 14870: grub2-probe 2024-04-17T07:33:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 14870: grub2-probe 2024-04-17T07:33:33.135 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 14870: grub2-probe 2024-04-17T07:33:33.135 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 14870: grub2-probe 2024-04-17T07:33:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 14919: grub2-probe 2024-04-17T07:33:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 14919: grub2-probe 2024-04-17T07:33:33.441 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 14919: grub2-probe 2024-04-17T07:33:33.441 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 14919: grub2-probe 2024-04-17T07:33:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 14967: grub2-probe 2024-04-17T07:33:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 14967: grub2-probe 2024-04-17T07:33:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 14967: grub2-probe 2024-04-17T07:33:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 14967: grub2-probe 2024-04-17T07:33:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 15015: grub2-probe 2024-04-17T07:33:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 15015: grub2-probe 2024-04-17T07:33:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 15015: grub2-probe 2024-04-17T07:33:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 15015: grub2-probe 2024-04-17T07:33:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 15063: grub2-probe 2024-04-17T07:33:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 15063: grub2-probe 2024-04-17T07:33:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 3 (pipe:[32407]) leaked on vgs invocation. Parent PID 15063: grub2-probe 2024-04-17T07:33:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:File descriptor 9 (pipe:[35105]) leaked on vgs invocation. Parent PID 15063: grub2-probe 2024-04-17T07:33:34.141 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-17T07:33:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-04-17T07:33:34.164 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-04-17T07:33:34.180 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.YzIlyf8REa 2024-04-17T07:33:34.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YzIlyf8REa 2024-04-17T07:33:34.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.YzIlyf8REa 2024-04-17T07:33:34.364 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.YzIlyf8REa is 6KB 2024-04-17T07:33:34.413 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.YzIlyf8REa 2024-04-17T07:33:34.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T07:33:34.500 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-17T07:33:34.501 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-17T07:33:34.501 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-17T07:33:34.501 INFO:teuthology.orchestra.run.smithi046.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-17T07:33:34.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:33:34.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-04-17T07:34:04.661 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-17T07:34:04.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-17T07:34:04.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:34:23.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-17T07:34:32.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-17T07:34:32.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:34:35.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-17T07:34:47.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-17T07:34:47.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:35:47.076 DEBUG:teuthology.orchestra.remote:timed out 2024-04-17T07:36:02.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-17T07:36:02.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:36:02.504 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-04-17T07:36:03.216 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-17T07:36:03.216 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-17T07:36:03.216 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-04-17T07:36:03.272 INFO:teuthology.orchestra.run.smithi046.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:36:03.273 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T07:36:03.273 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T07:36:03.273 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T07:36:04.274 DEBUG:teuthology.parallel:result is None 2024-04-17T07:36:04.275 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T07:36:04.284 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T07:36:04.285 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T07:36:04.292 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T07:36:04.295 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T07:36:04.344 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T07:36:04.351 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T07:36:04.357 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T07:36:04.357 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T07:36:04.365 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T07:36:04.368 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T07:36:04.435 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T07:36:04.442 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T07:36:04.443 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T07:36:04.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:36:04.461 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T07:36:04.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:36:04.478 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T07:36:04.485 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T07:36:04.504 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T07:36:04.548 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T07:36:04.561 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T07:36:04.570 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T07:36:04.596 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T07:36:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T07:36:04.722 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T07:36:04.727 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T07:36:04.735 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T07:36:04.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T07:36:04.742 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T07:36:04.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T07:36:04.847 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T07:36:04.861 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T07:36:04.861 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T07:36:04.868 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T07:36:04.871 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T07:36:04.921 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T07:36:05.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T07:36:05.120 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T07:36:05.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T07:36:05.314 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:36:05.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T07:36:05.414 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T07:36:05.460 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T07:36:05.518 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T07:36:05.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T07:36:05.653 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:36:05.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T07:36:05.726 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T07:36:05.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T07:36:05.826 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T07:36:05.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T07:36:05.958 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:36:05.958 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T07:36:06.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-04-17T07:36:06.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-04-17T07:36:06.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-04-17T07:36:06.079 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T07:36:06.114 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T07:36:06.172 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T07:36:06.691 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T07:36:06.699 INFO:teuthology.task.internal:Starting timer... 2024-04-17T07:36:06.699 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T07:36:06.710 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T07:36:06.718 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-17T07:36:06.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-04-17T07:36:06.839 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-04-17T07:36:06.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-04-17T07:36:06.919 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-04-17T07:36:06.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-04-17T07:36:06.987 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-04-17T07:36:06.989 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T07:36:06.989 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-04-17T07:36:07.049 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-04-17T07:36:07.050 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-04-17T07:36:07.068 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-04-17T07:36:07.068 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-04-17T07:36:07.090 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-04-17T07:36:07.091 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive'} 2024-04-17T07:36:07.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T07:36:07.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:36:07.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T07:36:07.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:36:07.213 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T07:36:07.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:36:07.245 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T07:36:07.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-04-17T07:36:07.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-04-17T07:36:07.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-04-17T07:36:07.426 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T07:36:07.435 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-17T07:36:07.441 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-17T07:36:07.579 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T07:36:07.594 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T07:36:07.596 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi053.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T07:42:41.268 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2024-04-17T07:42:41.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-17T07:42:41.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:42:41.423 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-04-17T07:42:41.588 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-17T07:42:41.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-17T07:42:41.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:42:41.695 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-04-17T07:42:41.767 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-17T07:42:41.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-17T07:42:41.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T07:42:41.872 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-17T07:42:41.943 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-17T07:42:41.943 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T07:42:41.957 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T07:42:41.958 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T07:42:41.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T07:42:41.966 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T07:42:41.967 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T07:42:41.969 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T07:42:41.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T07:42:42.006 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T07:42:42.021 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T07:42:42.028 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T07:42:42.043 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T07:42:42.067 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-04-17T07:42:42.070 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T07:42:42.080 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-04-17T07:42:42.084 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-04-17T07:42:42.097 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-04-17T07:42:42.104 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T07:42:42.118 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T07:42:42.124 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T07:42:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T07:42:42.138 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T07:42:42.173 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-04-17T07:42:42.176 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T07:42:42.176 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-04-17T07:42:42.176 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.176 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.177 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.177 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.186 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-17T07:42:42.190 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T07:42:42.190 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-17T07:42:42.190 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.190 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.190 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.190 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.222 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-04-17T07:42:42.264 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-04-17T07:42:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T07:42:42.375 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T07:42:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-04-17T07:42:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T07:42:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-04-17T07:42:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:42:42.524 INFO:teuthology.run_tasks:Running task install... 2024-04-17T07:42:42.532 DEBUG:teuthology.task.install:project ceph 2024-04-17T07:42:42.532 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-17T07:42:42.532 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-17T07:42:42.532 INFO:teuthology.task.install:Using flavor: default 2024-04-17T07:42:42.539 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T07:42:42.539 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-17T07:42:42.539 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T07:42:42.540 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:42:42.542 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T07:42:42.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:42:42.543 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T07:42:42.543 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:42:42.693 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T07:42:42.693 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T07:42:42.706 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T07:42:42.706 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T07:42:42.773 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T07:42:42.773 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T07:42:42.814 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T07:42:42.814 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:42:42.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T07:42:42.850 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T07:42:42.850 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T07:42:42.876 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T07:42:42.876 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:42:42.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T07:42:42.881 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T07:42:42.881 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:42:42.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T07:42:42.913 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T07:42:42.913 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T07:42:42.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T07:42:42.986 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T07:42:42.987 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T07:42:42.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T07:42:43.013 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T07:42:43.043 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-04-17T07:42:43.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-04-17T07:42:43.081 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T07:42:43.113 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-04-17T07:42:43.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-04-17T07:42:43.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T07:42:43.259 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T07:42:43.292 INFO:teuthology.orchestra.run.smithi053.stdout:74 files removed 2024-04-17T07:42:43.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T07:42:43.352 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-04-17T07:42:43.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-04-17T07:42:43.366 INFO:teuthology.orchestra.run.smithi062.stdout:74 files removed 2024-04-17T07:42:43.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T07:42:43.721 INFO:teuthology.orchestra.run.smithi046.stdout:74 files removed 2024-04-17T07:42:43.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T07:42:44.035 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 206 kB/s | 84 kB 00:00 2024-04-17T07:42:44.143 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 190 kB/s | 84 kB 00:00 2024-04-17T07:42:44.453 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-17T07:42:44.569 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-17T07:42:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 205 kB/s | 84 kB 00:00 2024-04-17T07:42:44.929 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-04-17T07:42:45.053 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-17T07:42:45.165 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-17T07:42:45.564 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-17T07:42:45.680 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-04-17T07:42:45.790 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-04-17T07:42:46.425 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-17T07:42:47.098 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2024-04-17T07:42:47.155 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-04-17T07:42:47.889 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-04-17T07:42:48.170 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-04-17T07:42:48.629 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-17T07:42:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-04-17T07:42:50.489 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-17T07:42:54.674 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-17T07:42:56.374 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-17T07:42:57.309 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-04-17T07:43:02.601 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-17T07:43:05.448 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T07:43:05.448 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T07:43:05.462 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T07:43:05.463 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T07:43:05.464 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T07:43:05.465 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T07:43:05.467 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T07:43:05.470 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T07:43:05.471 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T07:43:05.560 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-17T07:43:05.577 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-04-17T07:43:05.577 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T07:43:05.578 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T07:43:05.579 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T07:43:05.580 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T07:43:05.581 INFO:teuthology.orchestra.run.smithi062.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T07:43:05.582 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T07:43:05.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T07:43:05.584 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T07:43:05.585 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T07:43:05.586 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T07:43:05.587 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T07:43:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T07:43:05.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T07:43:05.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout:Install 150 Packages 2024-04-17T07:43:05.591 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 24 Packages 2024-04-17T07:43:05.592 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:05.592 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 229 M 2024-04-17T07:43:05.592 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-17T07:43:06.594 INFO:teuthology.orchestra.run.smithi062.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-17T07:43:06.778 INFO:teuthology.orchestra.run.smithi062.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 4.8 MB/s | 893 kB 00:00 2024-04-17T07:43:06.845 INFO:teuthology.orchestra.run.smithi062.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 2.1 MB/s | 145 kB 00:00 2024-04-17T07:43:07.037 INFO:teuthology.orchestra.run.smithi062.stdout:(4/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 8.8 MB/s | 5.4 MB 00:00 2024-04-17T07:43:07.104 INFO:teuthology.orchestra.run.smithi062.stdout:(5/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 9.1 MB/s | 2.4 MB 00:00 2024-04-17T07:43:07.171 INFO:teuthology.orchestra.run.smithi062.stdout:(6/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-17T07:43:07.588 INFO:teuthology.orchestra.run.smithi062.stdout:(7/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 9.7 MB/s | 4.7 MB 00:00 2024-04-17T07:43:08.223 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-04-17T07:43:08.430 INFO:teuthology.orchestra.run.smithi062.stdout:(8/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 13 MB/s | 17 MB 00:01 2024-04-17T07:43:08.496 INFO:teuthology.orchestra.run.smithi062.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 379 kB/s | 25 kB 00:00 2024-04-17T07:43:08.605 INFO:teuthology.orchestra.run.smithi062.stdout:(10/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 10 MB/s | 11 MB 00:01 2024-04-17T07:43:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.6 MB/s | 225 kB 00:00 2024-04-17T07:43:08.780 INFO:teuthology.orchestra.run.smithi062.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 465 kB/s | 27 kB 00:00 2024-04-17T07:43:08.814 INFO:teuthology.orchestra.run.smithi062.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 477 kB/s | 32 kB 00:00 2024-04-17T07:43:08.948 INFO:teuthology.orchestra.run.smithi062.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 5.5 MB/s | 748 kB 00:00 2024-04-17T07:43:09.023 INFO:teuthology.orchestra.run.smithi062.stdout:(15/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 2.1 MB/s | 163 kB 00:00 2024-04-17T07:43:09.098 INFO:teuthology.orchestra.run.smithi062.stdout:(16/174): librados-devel-19.0.0-3133.ga8c84f59. 1.6 MB/s | 126 kB 00:00 2024-04-17T07:43:09.182 INFO:teuthology.orchestra.run.smithi062.stdout:(17/174): libradosstriper1-19.0.0-3133.ga8c84f5 5.9 MB/s | 501 kB 00:00 2024-04-17T07:43:09.791 INFO:teuthology.orchestra.run.smithi062.stdout:(18/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 8.8 MB/s | 5.4 MB 00:00 2024-04-17T07:43:09.866 INFO:teuthology.orchestra.run.smithi062.stdout:(19/174): python3-ceph-argparse-19.0.0-3133.ga8 603 kB/s | 45 kB 00:00 2024-04-17T07:43:09.949 INFO:teuthology.orchestra.run.smithi062.stdout:(20/174): python3-ceph-common-19.0.0-3133.ga8c8 1.6 MB/s | 135 kB 00:00 2024-04-17T07:43:10.025 INFO:teuthology.orchestra.run.smithi062.stdout:(21/174): python3-cephfs-19.0.0-3133.ga8c84f59. 2.1 MB/s | 162 kB 00:00 2024-04-17T07:43:10.108 INFO:teuthology.orchestra.run.smithi062.stdout:(22/174): python3-rados-19.0.0-3133.ga8c84f59.e 3.8 MB/s | 321 kB 00:00 2024-04-17T07:43:10.192 INFO:teuthology.orchestra.run.smithi062.stdout:(23/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 3.5 MB/s | 299 kB 00:00 2024-04-17T07:43:10.267 INFO:teuthology.orchestra.run.smithi062.stdout:(24/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.3 MB/s | 99 kB 00:00 2024-04-17T07:43:10.334 INFO:teuthology.orchestra.run.smithi062.stdout:(25/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-17T07:43:10.363 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 915 kB/s | 19 MB 00:21 2024-04-17T07:43:10.735 INFO:teuthology.orchestra.run.smithi062.stdout:(26/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 7.7 MB/s | 3.1 MB 00:00 2024-04-17T07:43:10.819 INFO:teuthology.orchestra.run.smithi062.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-17T07:43:10.894 INFO:teuthology.orchestra.run.smithi062.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3133.g 456 kB/s | 34 kB 00:00 2024-04-17T07:43:10.961 INFO:teuthology.orchestra.run.smithi062.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 2.1 MB/s | 142 kB 00:00 2024-04-17T07:43:11.437 INFO:teuthology.orchestra.run.smithi062.stdout:(30/174): ceph-test-19.0.0-3133.ga8c84f59.el9.x 17 MB/s | 49 MB 00:02 2024-04-17T07:43:11.521 INFO:teuthology.orchestra.run.smithi062.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 6.3 MB/s | 3.5 MB 00:00 2024-04-17T07:43:11.604 INFO:teuthology.orchestra.run.smithi062.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3133.ga8 2.9 MB/s | 249 kB 00:00 2024-04-17T07:43:11.690 INFO:teuthology.orchestra.run.smithi062.stdout:(33/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 593 kB/s | 49 kB 00:00 2024-04-17T07:43:11.755 INFO:teuthology.orchestra.run.smithi062.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3133.ga 247 kB/s | 16 kB 00:00 2024-04-17T07:43:11.889 INFO:teuthology.orchestra.run.smithi062.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-04-17T07:43:12.032 INFO:teuthology.orchestra.run.smithi062.stdout:(36/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 992 kB/s | 273 kB 00:00 2024-04-17T07:43:12.073 INFO:teuthology.orchestra.run.smithi062.stdout:(37/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 4.1 MB/s | 763 kB 00:00 2024-04-17T07:43:12.098 INFO:teuthology.orchestra.run.smithi062.stdout:(38/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 273 kB/s | 18 kB 00:00 2024-04-17T07:43:12.298 INFO:teuthology.orchestra.run.smithi062.stdout:(39/174): ceph-common-19.0.0-3133.ga8c84f59.el9 3.6 MB/s | 21 MB 00:05 2024-04-17T07:43:12.457 INFO:teuthology.orchestra.run.smithi062.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 106 kB 00:00 2024-04-17T07:43:12.482 INFO:teuthology.orchestra.run.smithi062.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-04-17T07:43:12.507 INFO:teuthology.orchestra.run.smithi062.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 920 kB 00:00 2024-04-17T07:43:12.533 INFO:teuthology.orchestra.run.smithi062.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 307 kB 00:00 2024-04-17T07:43:12.558 INFO:teuthology.orchestra.run.smithi062.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-04-17T07:43:12.642 INFO:teuthology.orchestra.run.smithi062.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-17T07:43:12.667 INFO:teuthology.orchestra.run.smithi062.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 17 kB 00:00 2024-04-17T07:43:12.692 INFO:teuthology.orchestra.run.smithi062.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 76 kB/s | 10 kB 00:00 2024-04-17T07:43:12.717 INFO:teuthology.orchestra.run.smithi062.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-04-17T07:43:12.743 INFO:teuthology.orchestra.run.smithi062.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-04-17T07:43:12.768 INFO:teuthology.orchestra.run.smithi062.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-04-17T07:43:12.802 INFO:teuthology.orchestra.run.smithi062.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 578 kB 00:00 2024-04-17T07:43:12.827 INFO:teuthology.orchestra.run.smithi062.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 803 kB/s | 46 kB 00:00 2024-04-17T07:43:12.852 INFO:teuthology.orchestra.run.smithi062.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-04-17T07:43:12.878 INFO:teuthology.orchestra.run.smithi062.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-04-17T07:43:12.902 INFO:teuthology.orchestra.run.smithi062.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-04-17T07:43:12.928 INFO:teuthology.orchestra.run.smithi062.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-04-17T07:43:13.028 INFO:teuthology.orchestra.run.smithi062.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-17T07:43:13.050 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.2 MB 00:00 2024-04-17T07:43:13.070 INFO:teuthology.orchestra.run.smithi062.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 9.3 MB/s | 1.5 MB 00:00 2024-04-17T07:43:13.104 INFO:teuthology.orchestra.run.smithi062.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 817 kB 00:00 2024-04-17T07:43:13.120 INFO:teuthology.orchestra.run.smithi062.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 132 kB 00:00 2024-04-17T07:43:13.179 INFO:teuthology.orchestra.run.smithi062.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-04-17T07:43:13.213 INFO:teuthology.orchestra.run.smithi062.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 845 kB/s | 78 kB 00:00 2024-04-17T07:43:13.296 INFO:teuthology.orchestra.run.smithi062.stdout:(63/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-04-17T07:43:13.388 INFO:teuthology.orchestra.run.smithi062.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-04-17T07:43:13.413 INFO:teuthology.orchestra.run.smithi062.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 124 kB/s | 24 kB 00:00 2024-04-17T07:43:13.439 INFO:teuthology.orchestra.run.smithi062.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-04-17T07:43:13.464 INFO:teuthology.orchestra.run.smithi062.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-17T07:43:13.489 INFO:teuthology.orchestra.run.smithi062.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2024-04-17T07:43:13.573 INFO:teuthology.orchestra.run.smithi062.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-04-17T07:43:13.598 INFO:teuthology.orchestra.run.smithi062.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-04-17T07:43:13.615 INFO:teuthology.orchestra.run.smithi062.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-04-17T07:43:13.641 INFO:teuthology.orchestra.run.smithi062.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-04-17T07:43:13.666 INFO:teuthology.orchestra.run.smithi062.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-04-17T07:43:13.691 INFO:teuthology.orchestra.run.smithi062.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-17T07:43:13.716 INFO:teuthology.orchestra.run.smithi062.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-17T07:43:13.733 INFO:teuthology.orchestra.run.smithi062.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-04-17T07:43:13.758 INFO:teuthology.orchestra.run.smithi062.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-04-17T07:43:13.775 INFO:teuthology.orchestra.run.smithi062.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 730 kB/s | 43 kB 00:00 2024-04-17T07:43:14.142 INFO:teuthology.orchestra.run.smithi062.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-17T07:43:14.159 INFO:teuthology.orchestra.run.smithi062.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 108 kB/s | 41 kB 00:00 2024-04-17T07:43:14.176 INFO:teuthology.orchestra.run.smithi062.stdout:(81/174): python3-toml-0.10.2-6.el9.noarch.rpm 97 kB/s | 39 kB 00:00 2024-04-17T07:43:14.201 INFO:teuthology.orchestra.run.smithi062.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-04-17T07:43:14.218 INFO:teuthology.orchestra.run.smithi062.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 599 kB/s | 35 kB 00:00 2024-04-17T07:43:14.235 INFO:teuthology.orchestra.run.smithi062.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-17T07:43:14.260 INFO:teuthology.orchestra.run.smithi062.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-17T07:43:14.285 INFO:teuthology.orchestra.run.smithi062.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 841 kB/s | 56 kB 00:00 2024-04-17T07:43:14.311 INFO:teuthology.orchestra.run.smithi062.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-17T07:43:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 565 kB/s | 44 kB 00:00 2024-04-17T07:43:14.364 INFO:teuthology.orchestra.run.smithi062.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-17T07:43:14.389 INFO:teuthology.orchestra.run.smithi062.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-17T07:43:14.423 INFO:teuthology.orchestra.run.smithi062.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-17T07:43:14.456 INFO:teuthology.orchestra.run.smithi062.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-17T07:43:14.482 INFO:teuthology.orchestra.run.smithi062.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-04-17T07:43:14.499 INFO:teuthology.orchestra.run.smithi062.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-04-17T07:43:14.538 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-17T07:43:14.590 INFO:teuthology.orchestra.run.smithi062.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-04-17T07:43:14.616 INFO:teuthology.orchestra.run.smithi062.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-17T07:43:14.641 INFO:teuthology.orchestra.run.smithi062.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-17T07:43:14.666 INFO:teuthology.orchestra.run.smithi062.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-17T07:43:14.725 INFO:teuthology.orchestra.run.smithi062.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-04-17T07:43:14.751 INFO:teuthology.orchestra.run.smithi062.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 233 kB/s | 25 kB 00:00 2024-04-17T07:43:14.859 INFO:teuthology.orchestra.run.smithi062.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.2 MB/s | 245 kB 00:00 2024-04-17T07:43:14.884 INFO:teuthology.orchestra.run.smithi062.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.8 MB/s | 248 kB 00:00 2024-04-17T07:43:14.909 INFO:teuthology.orchestra.run.smithi062.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-17T07:43:14.941 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-17T07:43:15.293 INFO:teuthology.orchestra.run.smithi062.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 54 kB/s | 22 kB 00:00 2024-04-17T07:43:15.501 INFO:teuthology.orchestra.run.smithi062.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.9 MB/s | 4.1 MB 00:00 2024-04-17T07:43:15.677 INFO:teuthology.orchestra.run.smithi062.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-17T07:43:15.785 INFO:teuthology.orchestra.run.smithi062.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-17T07:43:15.911 INFO:teuthology.orchestra.run.smithi062.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 83 kB/s | 19 kB 00:00 2024-04-17T07:43:15.936 INFO:teuthology.orchestra.run.smithi062.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 712 kB/s | 309 kB 00:00 2024-04-17T07:43:15.995 INFO:teuthology.orchestra.run.smithi062.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-04-17T07:43:16.020 INFO:teuthology.orchestra.run.smithi062.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-04-17T07:43:16.087 INFO:teuthology.orchestra.run.smithi062.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-17T07:43:16.112 INFO:teuthology.orchestra.run.smithi062.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-17T07:43:16.145 INFO:teuthology.orchestra.run.smithi062.stdout:(114/174): libunwind-1.6.2-1.el9.x86_64.rpm 447 kB/s | 67 kB 00:00 2024-04-17T07:43:16.179 INFO:teuthology.orchestra.run.smithi062.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-17T07:43:16.229 INFO:teuthology.orchestra.run.smithi062.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-17T07:43:16.255 INFO:teuthology.orchestra.run.smithi062.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-17T07:43:16.280 INFO:teuthology.orchestra.run.smithi062.stdout:(118/174): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-17T07:43:16.305 INFO:teuthology.orchestra.run.smithi062.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-17T07:43:16.330 INFO:teuthology.orchestra.run.smithi062.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-17T07:43:16.364 INFO:teuthology.orchestra.run.smithi062.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-17T07:43:16.398 INFO:teuthology.orchestra.run.smithi062.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-04-17T07:43:16.440 INFO:teuthology.orchestra.run.smithi062.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-17T07:43:16.465 INFO:teuthology.orchestra.run.smithi062.stdout:(124/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-17T07:43:16.515 INFO:teuthology.orchestra.run.smithi062.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-17T07:43:16.543 INFO:teuthology.orchestra.run.smithi062.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-17T07:43:16.566 INFO:teuthology.orchestra.run.smithi062.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-17T07:43:16.591 INFO:teuthology.orchestra.run.smithi062.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-17T07:43:16.617 INFO:teuthology.orchestra.run.smithi062.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 305 kB/s | 23 kB 00:00 2024-04-17T07:43:16.642 INFO:teuthology.orchestra.run.smithi062.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-17T07:43:16.667 INFO:teuthology.orchestra.run.smithi062.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 259 kB/s | 20 kB 00:00 2024-04-17T07:43:16.718 INFO:teuthology.orchestra.run.smithi062.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-04-17T07:43:16.743 INFO:teuthology.orchestra.run.smithi062.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-17T07:43:16.785 INFO:teuthology.orchestra.run.smithi062.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-17T07:43:16.810 INFO:teuthology.orchestra.run.smithi062.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-04-17T07:43:16.835 INFO:teuthology.orchestra.run.smithi062.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-17T07:43:16.860 INFO:teuthology.orchestra.run.smithi062.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-17T07:43:16.886 INFO:teuthology.orchestra.run.smithi062.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-17T07:43:16.911 INFO:teuthology.orchestra.run.smithi062.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-17T07:43:16.936 INFO:teuthology.orchestra.run.smithi062.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-17T07:43:16.962 INFO:teuthology.orchestra.run.smithi062.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-17T07:43:16.987 INFO:teuthology.orchestra.run.smithi062.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-17T07:43:17.012 INFO:teuthology.orchestra.run.smithi062.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-17T07:43:17.037 INFO:teuthology.orchestra.run.smithi062.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-17T07:43:17.063 INFO:teuthology.orchestra.run.smithi062.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-17T07:43:17.088 INFO:teuthology.orchestra.run.smithi062.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-17T07:43:17.097 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-17T07:43:17.122 INFO:teuthology.orchestra.run.smithi062.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-17T07:43:17.139 INFO:teuthology.orchestra.run.smithi062.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-17T07:43:17.164 INFO:teuthology.orchestra.run.smithi062.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-17T07:43:17.281 INFO:teuthology.orchestra.run.smithi062.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-17T07:43:17.291 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-17T07:43:17.364 INFO:teuthology.orchestra.run.smithi062.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-04-17T07:43:17.431 INFO:teuthology.orchestra.run.smithi062.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-17T07:43:17.465 INFO:teuthology.orchestra.run.smithi062.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-17T07:43:17.507 INFO:teuthology.orchestra.run.smithi062.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-17T07:43:17.549 INFO:teuthology.orchestra.run.smithi062.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-17T07:43:17.599 INFO:teuthology.orchestra.run.smithi062.stdout:(156/174): librados2-19.0.0-3133.ga8c84f59.el9. 7.4 MB/s | 3.4 MB 00:00 2024-04-17T07:43:17.633 INFO:teuthology.orchestra.run.smithi062.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 856 kB/s | 71 kB 00:00 2024-04-17T07:43:17.666 INFO:teuthology.orchestra.run.smithi062.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 544 kB/s | 36 kB 00:00 2024-04-17T07:43:17.725 INFO:teuthology.orchestra.run.smithi062.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-04-17T07:43:17.750 INFO:teuthology.orchestra.run.smithi062.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-04-17T07:43:17.776 INFO:teuthology.orchestra.run.smithi062.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 459 kB/s | 23 kB 00:00 2024-04-17T07:43:17.801 INFO:teuthology.orchestra.run.smithi062.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-17T07:43:17.826 INFO:teuthology.orchestra.run.smithi062.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-17T07:43:17.852 INFO:teuthology.orchestra.run.smithi062.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-17T07:43:17.877 INFO:teuthology.orchestra.run.smithi062.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-17T07:43:17.902 INFO:teuthology.orchestra.run.smithi062.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 572 kB/s | 29 kB 00:00 2024-04-17T07:43:17.953 INFO:teuthology.orchestra.run.smithi062.stdout:(167/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-17T07:43:18.003 INFO:teuthology.orchestra.run.smithi062.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-04-17T07:43:18.045 INFO:teuthology.orchestra.run.smithi062.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 15 MB/s | 2.2 MB 00:00 2024-04-17T07:43:18.070 INFO:teuthology.orchestra.run.smithi062.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-04-17T07:43:18.095 INFO:teuthology.orchestra.run.smithi062.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 620 kB/s | 31 kB 00:00 2024-04-17T07:43:18.137 INFO:teuthology.orchestra.run.smithi062.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-04-17T07:43:18.179 INFO:teuthology.orchestra.run.smithi062.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 10 MB/s | 2.3 MB 00:00 2024-04-17T07:43:18.204 INFO:teuthology.orchestra.run.smithi062.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 683 kB/s | 74 kB 00:00 2024-04-17T07:43:18.211 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-17T07:43:18.211 INFO:teuthology.orchestra.run.smithi062.stdout:Total 18 MB/s | 229 MB 00:12 2024-04-17T07:43:18.605 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-17T07:43:18.907 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-17T07:43:18.907 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-17T07:43:18.928 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-17T07:43:20.754 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-17T07:43:20.754 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-17T07:43:23.301 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-17T07:43:24.308 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-17T07:43:26.347 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T07:43:26.348 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T07:43:26.372 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T07:43:26.374 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T07:43:26.375 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T07:43:26.377 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T07:43:26.380 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T07:43:26.384 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T07:43:26.385 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T07:43:26.480 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-04-17T07:43:26.503 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================================ 2024-04-17T07:43:26.503 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-04-17T07:43:26.503 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================================ 2024-04-17T07:43:26.503 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-04-17T07:43:26.503 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T07:43:26.503 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T07:43:26.504 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T07:43:26.505 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T07:43:26.506 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T07:43:26.507 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T07:43:26.508 INFO:teuthology.orchestra.run.smithi046.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T07:43:26.509 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T07:43:26.510 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T07:43:26.511 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T07:43:26.512 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T07:43:26.513 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T07:43:26.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T07:43:26.515 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T07:43:26.516 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:43:26.517 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-04-17T07:43:26.518 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================================ 2024-04-17T07:43:26.518 INFO:teuthology.orchestra.run.smithi046.stdout:Install 150 Packages 2024-04-17T07:43:26.518 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 24 Packages 2024-04-17T07:43:26.518 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:43:26.518 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 229 M 2024-04-17T07:43:26.518 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-04-17T07:43:27.082 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T07:43:27.083 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T07:43:27.097 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T07:43:27.098 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T07:43:27.100 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T07:43:27.101 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T07:43:27.103 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T07:43:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T07:43:27.107 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T07:43:27.192 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-04-17T07:43:27.209 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-04-17T07:43:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T07:43:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T07:43:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T07:43:27.212 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T07:43:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T07:43:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T07:43:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T07:43:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T07:43:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T07:43:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T07:43:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T07:43:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T07:43:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T07:43:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2024-04-17T07:43:27.223 INFO:teuthology.orchestra.run.smithi053.stdout:Install 150 Packages 2024-04-17T07:43:27.224 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 24 Packages 2024-04-17T07:43:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:43:27.224 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 229 M 2024-04-17T07:43:27.224 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-04-17T07:43:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-17T07:43:27.852 INFO:teuthology.orchestra.run.smithi053.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-17T07:43:27.888 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-17T07:43:28.017 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T07:43:28.043 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T07:43:28.068 INFO:teuthology.orchestra.run.smithi053.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 4.0 MB/s | 893 kB 00:00 2024-04-17T07:43:28.073 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T07:43:28.078 INFO:teuthology.orchestra.run.smithi046.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 2.6 MB/s | 893 kB 00:00 2024-04-17T07:43:28.083 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T07:43:28.119 INFO:teuthology.orchestra.run.smithi053.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 2.8 MB/s | 145 kB 00:00 2024-04-17T07:43:28.123 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T07:43:28.161 INFO:teuthology.orchestra.run.smithi046.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 1.7 MB/s | 145 kB 00:00 2024-04-17T07:43:28.177 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T07:43:28.211 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T07:43:28.236 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T07:43:28.279 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T07:43:28.552 INFO:teuthology.orchestra.run.smithi053.stdout:(4/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 5.5 MB/s | 2.4 MB 00:00 2024-04-17T07:43:28.626 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T07:43:28.670 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:43:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:43:28.803 INFO:teuthology.orchestra.run.smithi046.stdout:(4/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 3.7 MB/s | 2.4 MB 00:00 2024-04-17T07:43:28.804 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:43:28.830 INFO:teuthology.orchestra.run.smithi053.stdout:(5/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-04-17T07:43:28.880 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:43:28.925 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T07:43:28.938 INFO:teuthology.orchestra.run.smithi046.stdout:(5/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-04-17T07:43:28.970 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T07:43:29.144 INFO:teuthology.orchestra.run.smithi053.stdout:(6/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-04-17T07:43:29.150 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T07:43:29.209 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T07:43:29.238 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:43:29.271 INFO:teuthology.orchestra.run.smithi046.stdout:(6/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-04-17T07:43:29.342 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:43:29.371 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T07:43:29.435 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T07:43:29.466 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T07:43:29.489 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T07:43:29.505 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T07:43:29.665 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T07:43:29.686 INFO:teuthology.orchestra.run.smithi053.stdout:(7/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 5.5 MB/s | 4.7 MB 00:00 2024-04-17T07:43:29.695 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T07:43:29.735 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T07:43:29.845 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T07:43:29.946 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T07:43:29.997 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T07:43:30.072 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:43:30.124 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:43:30.129 INFO:teuthology.orchestra.run.smithi046.stdout:(7/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-04-17T07:43:30.352 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:43:30.458 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T07:43:30.553 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T07:43:30.586 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T07:43:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T07:43:30.649 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T07:43:30.728 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T07:43:30.787 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T07:43:30.824 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T07:43:30.900 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T07:43:30.940 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T07:43:30.983 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T07:43:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:(8/174): ceph-common-19.0.0-3133.ga8c84f59.el9. 5.9 MB/s | 21 MB 00:03 2024-04-17T07:43:31.015 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T07:43:31.043 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T07:43:31.071 INFO:teuthology.orchestra.run.smithi046.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 306 kB/s | 25 kB 00:00 2024-04-17T07:43:31.116 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T07:43:31.148 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T07:43:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T07:43:31.220 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T07:43:31.295 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T07:43:31.352 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T07:43:31.430 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T07:43:31.581 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T07:43:31.639 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T07:43:32.207 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T07:43:32.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T07:43:32.353 INFO:teuthology.orchestra.run.smithi053.stdout:(8/174): ceph-common-19.0.0-3133.ga8c84f59.el9. 4.5 MB/s | 21 MB 00:04 2024-04-17T07:43:32.376 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T07:43:32.419 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T07:43:32.428 INFO:teuthology.orchestra.run.smithi053.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 336 kB/s | 25 kB 00:00 2024-04-17T07:43:32.429 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T07:43:32.457 INFO:teuthology.orchestra.run.smithi046.stdout:(10/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 4.6 MB/s | 11 MB 00:02 2024-04-17T07:43:32.459 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T07:43:32.508 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T07:43:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T07:43:32.571 INFO:teuthology.orchestra.run.smithi046.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 1.9 MB/s | 225 kB 00:00 2024-04-17T07:43:32.575 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T07:43:32.620 INFO:teuthology.orchestra.run.smithi053.stdout:(10/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 4.8 MB/s | 17 MB 00:03 2024-04-17T07:43:32.655 INFO:teuthology.orchestra.run.smithi046.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 331 kB/s | 27 kB 00:00 2024-04-17T07:43:32.729 INFO:teuthology.orchestra.run.smithi053.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.0 MB/s | 225 kB 00:00 2024-04-17T07:43:32.755 INFO:teuthology.orchestra.run.smithi046.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 320 kB/s | 32 kB 00:00 2024-04-17T07:43:32.820 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T07:43:32.829 INFO:teuthology.orchestra.run.smithi053.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 270 kB/s | 27 kB 00:00 2024-04-17T07:43:32.938 INFO:teuthology.orchestra.run.smithi053.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 292 kB/s | 32 kB 00:00 2024-04-17T07:43:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T07:43:32.981 INFO:teuthology.orchestra.run.smithi046.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 3.3 MB/s | 748 kB 00:00 2024-04-17T07:43:33.036 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T07:43:33.072 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T07:43:33.113 INFO:teuthology.orchestra.run.smithi053.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 4.2 MB/s | 748 kB 00:00 2024-04-17T07:43:33.181 INFO:teuthology.orchestra.run.smithi053.stdout:(15/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 2.4 MB/s | 163 kB 00:00 2024-04-17T07:43:33.218 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T07:43:33.248 INFO:teuthology.orchestra.run.smithi053.stdout:(16/174): librados-devel-19.0.0-3133.ga8c84f59. 1.9 MB/s | 126 kB 00:00 2024-04-17T07:43:33.264 INFO:teuthology.orchestra.run.smithi046.stdout:(15/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 4.2 MB/s | 17 MB 00:03 2024-04-17T07:43:33.284 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T07:43:33.289 INFO:teuthology.orchestra.run.smithi046.stdout:(16/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 531 kB/s | 163 kB 00:00 2024-04-17T07:43:33.365 INFO:teuthology.orchestra.run.smithi046.stdout:(17/174): librados-devel-19.0.0-3133.ga8c84f59. 1.2 MB/s | 126 kB 00:00 2024-04-17T07:43:33.406 INFO:teuthology.orchestra.run.smithi053.stdout:(17/174): libradosstriper1-19.0.0-3133.ga8c84f5 3.1 MB/s | 501 kB 00:00 2024-04-17T07:43:33.473 INFO:teuthology.orchestra.run.smithi046.stdout:(18/174): libradosstriper1-19.0.0-3133.ga8c84f5 2.7 MB/s | 501 kB 00:00 2024-04-17T07:43:33.537 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T07:43:33.582 INFO:teuthology.orchestra.run.smithi046.stdout:(19/174): python3-ceph-argparse-19.0.0-3133.ga8 421 kB/s | 45 kB 00:00 2024-04-17T07:43:33.699 INFO:teuthology.orchestra.run.smithi046.stdout:(20/174): python3-ceph-common-19.0.0-3133.ga8c8 1.1 MB/s | 135 kB 00:00 2024-04-17T07:43:33.776 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T07:43:33.816 INFO:teuthology.orchestra.run.smithi046.stdout:(21/174): python3-cephfs-19.0.0-3133.ga8c84f59. 1.4 MB/s | 162 kB 00:00 2024-04-17T07:43:33.958 INFO:teuthology.orchestra.run.smithi046.stdout:(22/174): python3-rados-19.0.0-3133.ga8c84f59.e 2.2 MB/s | 321 kB 00:00 2024-04-17T07:43:33.961 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T07:43:34.073 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T07:43:34.074 INFO:teuthology.orchestra.run.smithi046.stdout:(23/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 2.5 MB/s | 299 kB 00:00 2024-04-17T07:43:34.122 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T07:43:34.163 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T07:43:34.183 INFO:teuthology.orchestra.run.smithi046.stdout:(24/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 924 kB/s | 99 kB 00:00 2024-04-17T07:43:34.207 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T07:43:34.249 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T07:43:34.292 INFO:teuthology.orchestra.run.smithi046.stdout:(25/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 793 kB/s | 85 kB 00:00 2024-04-17T07:43:34.394 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T07:43:34.444 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T07:43:34.474 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T07:43:34.516 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T07:43:34.616 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T07:43:34.657 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T07:43:34.697 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T07:43:34.698 INFO:teuthology.orchestra.run.smithi053.stdout:(18/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 4.2 MB/s | 5.4 MB 00:01 2024-04-17T07:43:34.723 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T07:43:34.756 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T07:43:34.815 INFO:teuthology.orchestra.run.smithi053.stdout:(19/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 2.1 MB/s | 11 MB 00:05 2024-04-17T07:43:34.825 INFO:teuthology.orchestra.run.smithi046.stdout:(26/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 3.7 MB/s | 5.4 MB 00:01 2024-04-17T07:43:34.832 INFO:teuthology.orchestra.run.smithi053.stdout:(20/174): python3-ceph-argparse-19.0.0-3133.ga8 338 kB/s | 45 kB 00:00 2024-04-17T07:43:34.834 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T07:43:34.907 INFO:teuthology.orchestra.run.smithi053.stdout:(21/174): python3-ceph-common-19.0.0-3133.ga8c8 1.4 MB/s | 135 kB 00:00 2024-04-17T07:43:34.933 INFO:teuthology.orchestra.run.smithi053.stdout:(22/174): python3-cephfs-19.0.0-3133.ga8c84f59. 1.6 MB/s | 162 kB 00:00 2024-04-17T07:43:34.942 INFO:teuthology.orchestra.run.smithi046.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.4 MB/s | 170 kB 00:00 2024-04-17T07:43:34.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T07:43:35.008 INFO:teuthology.orchestra.run.smithi053.stdout:(23/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 3.9 MB/s | 299 kB 00:00 2024-04-17T07:43:35.017 INFO:teuthology.orchestra.run.smithi046.stdout:(28/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 4.3 MB/s | 3.1 MB 00:00 2024-04-17T07:43:35.033 INFO:teuthology.orchestra.run.smithi053.stdout:(24/174): python3-rados-19.0.0-3133.ga8c84f59.e 2.5 MB/s | 321 kB 00:00 2024-04-17T07:43:35.051 INFO:teuthology.orchestra.run.smithi046.stdout:(29/174): ceph-grafana-dashboards-19.0.0-3133.g 316 kB/s | 34 kB 00:00 2024-04-17T07:43:35.070 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T07:43:35.075 INFO:teuthology.orchestra.run.smithi053.stdout:(25/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.5 MB/s | 99 kB 00:00 2024-04-17T07:43:35.100 INFO:teuthology.orchestra.run.smithi053.stdout:(26/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-17T07:43:35.101 INFO:teuthology.orchestra.run.smithi046.stdout:(30/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.7 MB/s | 142 kB 00:00 2024-04-17T07:43:35.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T07:43:35.173 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T07:43:35.217 INFO:teuthology.orchestra.run.smithi053.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.4 MB/s | 170 kB 00:00 2024-04-17T07:43:35.236 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T07:43:35.293 INFO:teuthology.orchestra.run.smithi053.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3133.g 456 kB/s | 34 kB 00:00 2024-04-17T07:43:35.418 INFO:teuthology.orchestra.run.smithi053.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.1 MB/s | 142 kB 00:00 2024-04-17T07:43:35.434 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T07:43:35.486 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T07:43:35.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T07:43:35.675 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T07:43:35.707 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T07:43:35.735 INFO:teuthology.orchestra.run.smithi053.stdout:(30/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 4.7 MB/s | 3.1 MB 00:00 2024-04-17T07:43:35.737 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T07:43:35.766 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T07:43:35.800 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T07:43:35.840 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T07:43:35.879 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T07:43:35.909 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T07:43:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T07:43:36.028 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T07:43:36.061 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T07:43:36.103 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T07:43:36.110 INFO:teuthology.orchestra.run.smithi046.stdout:(31/174): ceph-test-19.0.0-3133.ga8c84f59.el9.x 9.8 MB/s | 49 MB 00:05 2024-04-17T07:43:36.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T07:43:36.219 INFO:teuthology.orchestra.run.smithi046.stdout:(32/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 3.0 MB/s | 3.5 MB 00:01 2024-04-17T07:43:36.232 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T07:43:36.252 INFO:teuthology.orchestra.run.smithi046.stdout:(33/174): ceph-mgr-modules-core-19.0.0-3133.ga8 1.7 MB/s | 249 kB 00:00 2024-04-17T07:43:36.294 INFO:teuthology.orchestra.run.smithi046.stdout:(34/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 667 kB/s | 49 kB 00:00 2024-04-17T07:43:36.321 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T07:43:36.328 INFO:teuthology.orchestra.run.smithi046.stdout:(35/174): ceph-prometheus-alerts-19.0.0-3133.ga 221 kB/s | 16 kB 00:00 2024-04-17T07:43:36.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T07:43:36.416 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T07:43:36.460 INFO:teuthology.orchestra.run.smithi053.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 3.4 MB/s | 3.5 MB 00:01 2024-04-17T07:43:36.462 INFO:teuthology.orchestra.run.smithi046.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-04-17T07:43:36.464 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T07:43:36.518 INFO:teuthology.orchestra.run.smithi046.stdout:(37/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.2 MB/s | 273 kB 00:00 2024-04-17T07:43:36.527 INFO:teuthology.orchestra.run.smithi053.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3133.ga8 3.7 MB/s | 249 kB 00:00 2024-04-17T07:43:36.545 INFO:teuthology.orchestra.run.smithi046.stdout:(38/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 221 kB/s | 18 kB 00:00 2024-04-17T07:43:36.577 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T07:43:36.577 INFO:teuthology.orchestra.run.smithi053.stdout:(33/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 989 kB/s | 49 kB 00:00 2024-04-17T07:43:36.636 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T07:43:36.637 INFO:teuthology.orchestra.run.smithi053.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3133.ga 281 kB/s | 16 kB 00:00 2024-04-17T07:43:36.704 INFO:teuthology.orchestra.run.smithi046.stdout:(39/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 2.0 MB/s | 763 kB 00:00 2024-04-17T07:43:36.736 INFO:teuthology.orchestra.run.smithi053.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 7.4 MB/s | 7.4 MB 00:01 2024-04-17T07:43:36.762 INFO:teuthology.orchestra.run.smithi053.stdout:(36/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 2.1 MB/s | 273 kB 00:00 2024-04-17T07:43:36.804 INFO:teuthology.orchestra.run.smithi053.stdout:(37/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 439 kB/s | 18 kB 00:00 2024-04-17T07:43:36.821 INFO:teuthology.orchestra.run.smithi046.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 920 kB 00:00 2024-04-17T07:43:36.846 INFO:teuthology.orchestra.run.smithi053.stdout:(38/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 6.8 MB/s | 763 kB 00:00 2024-04-17T07:43:36.863 INFO:teuthology.orchestra.run.smithi046.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2024-04-17T07:43:36.888 INFO:teuthology.orchestra.run.smithi046.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 587 kB/s | 106 kB 00:00 2024-04-17T07:43:36.922 INFO:teuthology.orchestra.run.smithi046.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.2 MB/s | 307 kB 00:00 2024-04-17T07:43:36.955 INFO:teuthology.orchestra.run.smithi046.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 501 kB/s | 32 kB 00:00 2024-04-17T07:43:36.988 INFO:teuthology.orchestra.run.smithi053.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-04-17T07:43:37.038 INFO:teuthology.orchestra.run.smithi053.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.4 MB/s | 920 kB 00:00 2024-04-17T07:43:37.055 INFO:teuthology.orchestra.run.smithi046.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 133 kB/s | 17 kB 00:00 2024-04-17T07:43:37.063 INFO:teuthology.orchestra.run.smithi053.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-04-17T07:43:37.088 INFO:teuthology.orchestra.run.smithi053.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-17T07:43:37.114 INFO:teuthology.orchestra.run.smithi053.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2024-04-17T07:43:37.131 INFO:teuthology.orchestra.run.smithi046.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-17T07:43:37.156 INFO:teuthology.orchestra.run.smithi046.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 53 kB/s | 10 kB 00:00 2024-04-17T07:43:37.191 INFO:teuthology.orchestra.run.smithi046.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-04-17T07:43:37.197 INFO:teuthology.orchestra.run.smithi053.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-17T07:43:37.223 INFO:teuthology.orchestra.run.smithi053.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-04-17T07:43:37.223 INFO:teuthology.orchestra.run.smithi046.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 687 kB/s | 44 kB 00:00 2024-04-17T07:43:37.240 INFO:teuthology.orchestra.run.smithi053.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 240 kB/s | 10 kB 00:00 2024-04-17T07:43:37.265 INFO:teuthology.orchestra.run.smithi053.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-04-17T07:43:37.265 INFO:teuthology.orchestra.run.smithi046.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.1 MB/s | 661 kB 00:00 2024-04-17T07:43:37.290 INFO:teuthology.orchestra.run.smithi053.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-04-17T07:43:37.307 INFO:teuthology.orchestra.run.smithi046.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.1 MB/s | 578 kB 00:00 2024-04-17T07:43:37.315 INFO:teuthology.orchestra.run.smithi053.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-17T07:43:37.340 INFO:teuthology.orchestra.run.smithi046.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 675 kB/s | 46 kB 00:00 2024-04-17T07:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-04-17T07:43:37.358 INFO:teuthology.orchestra.run.smithi053.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-17T07:43:37.365 INFO:teuthology.orchestra.run.smithi046.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 282 kB 00:00 2024-04-17T07:43:37.383 INFO:teuthology.orchestra.run.smithi053.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-04-17T07:43:37.399 INFO:teuthology.orchestra.run.smithi046.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-04-17T07:43:37.408 INFO:teuthology.orchestra.run.smithi053.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-17T07:43:37.424 INFO:teuthology.orchestra.run.smithi046.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 616 kB/s | 34 kB 00:00 2024-04-17T07:43:37.434 INFO:teuthology.orchestra.run.smithi053.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-04-17T07:43:37.435 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T07:43:37.459 INFO:teuthology.orchestra.run.smithi053.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-04-17T07:43:37.482 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T07:43:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.7 MB/s | 217 kB 00:00 2024-04-17T07:43:37.515 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T07:43:37.559 INFO:teuthology.orchestra.run.smithi053.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-17T07:43:37.565 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T07:43:37.610 INFO:teuthology.orchestra.run.smithi053.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-04-17T07:43:37.643 INFO:teuthology.orchestra.run.smithi053.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 9.8 MB/s | 817 kB 00:00 2024-04-17T07:43:37.657 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T07:43:37.658 INFO:teuthology.orchestra.run.smithi046.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-17T07:43:37.668 INFO:teuthology.orchestra.run.smithi053.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-17T07:43:37.691 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T07:43:37.725 INFO:teuthology.orchestra.run.smithi046.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 5.3 MB/s | 1.5 MB 00:00 2024-04-17T07:43:37.758 INFO:teuthology.orchestra.run.smithi046.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 817 kB 00:00 2024-04-17T07:43:37.777 INFO:teuthology.orchestra.run.smithi053.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-04-17T07:43:37.784 INFO:teuthology.orchestra.run.smithi046.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 132 kB 00:00 2024-04-17T07:43:37.811 INFO:teuthology.orchestra.run.smithi053.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-04-17T07:43:37.836 INFO:teuthology.orchestra.run.smithi053.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-17T07:43:37.851 INFO:teuthology.orchestra.run.smithi046.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-17T07:43:37.876 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T07:43:37.905 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T07:43:37.964 INFO:teuthology.orchestra.run.smithi053.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 5.8 MB 00:00 2024-04-17T07:43:37.989 INFO:teuthology.orchestra.run.smithi053.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 162 kB/s | 24 kB 00:00 2024-04-17T07:43:37.994 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T07:43:38.015 INFO:teuthology.orchestra.run.smithi053.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 360 kB/s | 18 kB 00:00 2024-04-17T07:43:38.040 INFO:teuthology.orchestra.run.smithi053.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-17T07:43:38.043 INFO:teuthology.orchestra.run.smithi046.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.7 MB 00:00 2024-04-17T07:43:38.065 INFO:teuthology.orchestra.run.smithi053.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-04-17T07:43:38.085 INFO:teuthology.orchestra.run.smithi046.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 264 kB/s | 78 kB 00:00 2024-04-17T07:43:38.191 INFO:teuthology.orchestra.run.smithi053.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-04-17T07:43:38.216 INFO:teuthology.orchestra.run.smithi053.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-04-17T07:43:38.227 INFO:teuthology.orchestra.run.smithi046.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-04-17T07:43:38.241 INFO:teuthology.orchestra.run.smithi053.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-17T07:43:38.243 INFO:teuthology.orchestra.run.smithi046.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 132 kB/s | 24 kB 00:00 2024-04-17T07:43:38.260 INFO:teuthology.orchestra.run.smithi046.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-04-17T07:43:38.267 INFO:teuthology.orchestra.run.smithi053.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-17T07:43:38.292 INFO:teuthology.orchestra.run.smithi053.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-04-17T07:43:38.294 INFO:teuthology.orchestra.run.smithi046.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 32 kB 00:00 2024-04-17T07:43:38.317 INFO:teuthology.orchestra.run.smithi053.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-04-17T07:43:38.342 INFO:teuthology.orchestra.run.smithi053.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-17T07:43:38.353 INFO:teuthology.orchestra.run.smithi046.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 144 kB 00:00 2024-04-17T07:43:38.368 INFO:teuthology.orchestra.run.smithi053.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-17T07:43:38.393 INFO:teuthology.orchestra.run.smithi053.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-04-17T07:43:38.418 INFO:teuthology.orchestra.run.smithi053.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-04-17T07:43:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-04-17T07:43:38.520 INFO:teuthology.orchestra.run.smithi046.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-04-17T07:43:38.545 INFO:teuthology.orchestra.run.smithi046.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 992 kB/s | 174 kB 00:00 2024-04-17T07:43:38.595 INFO:teuthology.orchestra.run.smithi046.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-04-17T07:43:38.602 INFO:teuthology.orchestra.run.smithi053.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 213 kB/s | 39 kB 00:00 2024-04-17T07:43:38.620 INFO:teuthology.orchestra.run.smithi046.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 41 kB 00:00 2024-04-17T07:43:38.654 INFO:teuthology.orchestra.run.smithi046.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 238 kB 00:00 2024-04-17T07:43:38.687 INFO:teuthology.orchestra.run.smithi046.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-04-17T07:43:38.688 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T07:43:38.712 INFO:teuthology.orchestra.run.smithi046.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2024-04-17T07:43:38.738 INFO:teuthology.orchestra.run.smithi046.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 574 kB/s | 47 kB 00:00 2024-04-17T07:43:38.769 INFO:teuthology.orchestra.run.smithi053.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-04-17T07:43:38.788 INFO:teuthology.orchestra.run.smithi046.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 581 kB/s | 43 kB 00:00 2024-04-17T07:43:38.803 INFO:teuthology.orchestra.run.smithi053.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 205 kB/s | 41 kB 00:00 2024-04-17T07:43:38.808 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T07:43:38.836 INFO:teuthology.orchestra.run.smithi053.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 220 kB 00:00 2024-04-17T07:43:38.862 INFO:teuthology.orchestra.run.smithi053.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 599 kB/s | 35 kB 00:00 2024-04-17T07:43:38.887 INFO:teuthology.orchestra.run.smithi053.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 923 kB/s | 46 kB 00:00 2024-04-17T07:43:38.912 INFO:teuthology.orchestra.run.smithi053.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-04-17T07:43:38.930 INFO:teuthology.orchestra.run.smithi046.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 203 kB/s | 39 kB 00:00 2024-04-17T07:43:38.938 INFO:teuthology.orchestra.run.smithi053.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-17T07:43:38.963 INFO:teuthology.orchestra.run.smithi053.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-17T07:43:38.988 INFO:teuthology.orchestra.run.smithi053.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-04-17T07:43:39.013 INFO:teuthology.orchestra.run.smithi053.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-17T07:43:39.039 INFO:teuthology.orchestra.run.smithi053.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-17T07:43:39.072 INFO:teuthology.orchestra.run.smithi053.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-04-17T07:43:39.098 INFO:teuthology.orchestra.run.smithi053.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-17T07:43:39.123 INFO:teuthology.orchestra.run.smithi053.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-17T07:43:39.130 INFO:teuthology.orchestra.run.smithi046.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-17T07:43:39.148 INFO:teuthology.orchestra.run.smithi053.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-17T07:43:39.155 INFO:teuthology.orchestra.run.smithi046.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 117 kB/s | 41 kB 00:00 2024-04-17T07:43:39.189 INFO:teuthology.orchestra.run.smithi046.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 893 kB/s | 220 kB 00:00 2024-04-17T07:43:39.214 INFO:teuthology.orchestra.run.smithi046.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 843 kB/s | 46 kB 00:00 2024-04-17T07:43:39.239 INFO:teuthology.orchestra.run.smithi046.stdout:(84/174): qatlib-service-24.02.0-1.el9.x86_64.r 433 kB/s | 35 kB 00:00 2024-04-17T07:43:39.265 INFO:teuthology.orchestra.run.smithi046.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-17T07:43:39.290 INFO:teuthology.orchestra.run.smithi046.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 786 kB/s | 56 kB 00:00 2024-04-17T07:43:39.290 INFO:teuthology.orchestra.run.smithi053.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-17T07:43:39.315 INFO:teuthology.orchestra.run.smithi046.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-04-17T07:43:39.340 INFO:teuthology.orchestra.run.smithi046.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 599 kB/s | 44 kB 00:00 2024-04-17T07:43:39.341 INFO:teuthology.orchestra.run.smithi053.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-17T07:43:39.365 INFO:teuthology.orchestra.run.smithi046.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-17T07:43:39.374 INFO:teuthology.orchestra.run.smithi053.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-04-17T07:43:39.399 INFO:teuthology.orchestra.run.smithi053.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-17T07:43:39.407 INFO:teuthology.orchestra.run.smithi046.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-04-17T07:43:39.474 INFO:teuthology.orchestra.run.smithi046.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.1 MB 00:00 2024-04-17T07:43:39.499 INFO:teuthology.orchestra.run.smithi046.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 772 kB/s | 103 kB 00:00 2024-04-17T07:43:39.508 INFO:teuthology.orchestra.run.smithi053.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-04-17T07:43:39.533 INFO:teuthology.orchestra.run.smithi053.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 190 kB/s | 25 kB 00:00 2024-04-17T07:43:39.541 INFO:teuthology.orchestra.run.smithi046.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 3.0 MB/s | 124 kB 00:00 2024-04-17T07:43:39.567 INFO:teuthology.orchestra.run.smithi046.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-04-17T07:43:39.709 INFO:teuthology.orchestra.run.smithi046.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 905 kB/s | 187 kB 00:00 2024-04-17T07:43:39.742 INFO:teuthology.orchestra.run.smithi046.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-17T07:43:39.776 INFO:teuthology.orchestra.run.smithi046.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-17T07:43:39.826 INFO:teuthology.orchestra.run.smithi046.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-17T07:43:39.852 INFO:teuthology.orchestra.run.smithi046.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-17T07:43:39.885 INFO:teuthology.orchestra.run.smithi046.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 235 kB/s | 25 kB 00:00 2024-04-17T07:43:40.002 INFO:teuthology.orchestra.run.smithi046.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.6 MB/s | 245 kB 00:00 2024-04-17T07:43:40.017 INFO:teuthology.orchestra.run.smithi053.stdout:(100/174): ceph-test-19.0.0-3133.ga8c84f59.el9. 6.5 MB/s | 49 MB 00:07 2024-04-17T07:43:40.042 INFO:teuthology.orchestra.run.smithi053.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 482 kB/s | 245 kB 00:00 2024-04-17T07:43:40.061 INFO:teuthology.orchestra.run.smithi046.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-04-17T07:43:40.084 INFO:teuthology.orchestra.run.smithi053.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-17T07:43:40.086 INFO:teuthology.orchestra.run.smithi046.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.2 MB/s | 248 kB 00:00 2024-04-17T07:43:40.121 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T07:43:40.185 INFO:teuthology.orchestra.run.smithi053.stdout:(103/174): openssl-devel-3.0.7-27.el9.x86_64.rp 6.1 MB/s | 4.1 MB 00:00 2024-04-17T07:43:40.198 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T07:43:40.210 INFO:teuthology.orchestra.run.smithi053.stdout:(104/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.3 MB/s | 248 kB 00:00 2024-04-17T07:43:40.229 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T07:43:40.260 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T07:43:40.287 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T07:43:40.368 INFO:teuthology.orchestra.run.smithi053.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-04-17T07:43:40.477 INFO:teuthology.orchestra.run.smithi053.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-17T07:43:40.502 INFO:teuthology.orchestra.run.smithi053.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-04-17T07:43:40.530 INFO:teuthology.orchestra.run.smithi053.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 375 kB/s | 19 kB 00:00 2024-04-17T07:43:40.629 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T07:43:40.650 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T07:43:40.739 INFO:teuthology.orchestra.run.smithi053.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-17T07:43:40.742 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T07:43:40.755 INFO:teuthology.orchestra.run.smithi053.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 115 kB/s | 26 kB 00:00 2024-04-17T07:43:40.772 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T07:43:40.795 INFO:teuthology.orchestra.run.smithi046.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.3 MB/s | 4.1 MB 00:00 2024-04-17T07:43:40.809 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T07:43:40.814 INFO:teuthology.orchestra.run.smithi053.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 836 kB/s | 49 kB 00:00 2024-04-17T07:43:40.819 INFO:teuthology.orchestra.run.smithi046.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-04-17T07:43:40.830 INFO:teuthology.orchestra.run.smithi053.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 900 kB/s | 67 kB 00:00 2024-04-17T07:43:40.864 INFO:teuthology.orchestra.run.smithi053.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-17T07:43:40.906 INFO:teuthology.orchestra.run.smithi053.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-17T07:43:40.932 INFO:teuthology.orchestra.run.smithi053.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 8.2 MB/s | 548 kB 00:00 2024-04-17T07:43:40.948 INFO:teuthology.orchestra.run.smithi053.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-17T07:43:40.982 INFO:teuthology.orchestra.run.smithi053.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 645 kB/s | 32 kB 00:00 2024-04-17T07:43:41.007 INFO:teuthology.orchestra.run.smithi053.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-04-17T07:43:41.032 INFO:teuthology.orchestra.run.smithi053.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-17T07:43:41.058 INFO:teuthology.orchestra.run.smithi053.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-17T07:43:41.083 INFO:teuthology.orchestra.run.smithi053.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-04-17T07:43:41.108 INFO:teuthology.orchestra.run.smithi053.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 713 kB/s | 35 kB 00:00 2024-04-17T07:43:41.133 INFO:teuthology.orchestra.run.smithi053.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-04-17T07:43:41.167 INFO:teuthology.orchestra.run.smithi053.stdout:(124/174): protobuf-compiler-3.14.0-13.el9.x86_ 879 kB/s | 863 kB 00:00 2024-04-17T07:43:41.171 INFO:teuthology.orchestra.run.smithi046.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 883 kB/s | 309 kB 00:00 2024-04-17T07:43:41.222 INFO:teuthology.orchestra.run.smithi046.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 399 kB/s | 19 kB 00:00 2024-04-17T07:43:41.226 INFO:teuthology.orchestra.run.smithi053.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 18 MB/s | 2.0 MB 00:00 2024-04-17T07:43:41.251 INFO:teuthology.orchestra.run.smithi053.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-17T07:43:41.255 INFO:teuthology.orchestra.run.smithi046.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-17T07:43:41.276 INFO:teuthology.orchestra.run.smithi053.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 101 kB/s | 11 kB 00:00 2024-04-17T07:43:41.297 INFO:teuthology.orchestra.run.smithi046.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 635 kB/s | 26 kB 00:00 2024-04-17T07:43:41.301 INFO:teuthology.orchestra.run.smithi053.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 355 kB/s | 18 kB 00:00 2024-04-17T07:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T07:43:41.326 INFO:teuthology.orchestra.run.smithi053.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 309 kB/s | 23 kB 00:00 2024-04-17T07:43:41.339 INFO:teuthology.orchestra.run.smithi046.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-04-17T07:43:41.343 INFO:teuthology.orchestra.run.smithi053.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-04-17T07:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T07:43:41.377 INFO:teuthology.orchestra.run.smithi053.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-17T07:43:41.411 INFO:teuthology.orchestra.run.smithi053.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-17T07:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T07:43:41.423 INFO:teuthology.orchestra.run.smithi046.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 816 kB/s | 67 kB 00:00 2024-04-17T07:43:41.436 INFO:teuthology.orchestra.run.smithi053.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-04-17T07:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T07:43:41.461 INFO:teuthology.orchestra.run.smithi053.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 963 kB/s | 79 kB 00:00 2024-04-17T07:43:41.486 INFO:teuthology.orchestra.run.smithi053.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-17T07:43:41.506 INFO:teuthology.orchestra.run.smithi046.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-17T07:43:41.511 INFO:teuthology.orchestra.run.smithi053.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-17T07:43:41.528 INFO:teuthology.orchestra.run.smithi053.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 247 kB/s | 16 kB 00:00 2024-04-17T07:43:41.562 INFO:teuthology.orchestra.run.smithi053.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-17T07:43:41.579 INFO:teuthology.orchestra.run.smithi053.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-17T07:43:41.596 INFO:teuthology.orchestra.run.smithi053.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-04-17T07:43:41.621 INFO:teuthology.orchestra.run.smithi053.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-17T07:43:41.643 INFO:teuthology.orchestra.run.smithi053.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 770 kB/s | 36 kB 00:00 2024-04-17T07:43:41.663 INFO:teuthology.orchestra.run.smithi053.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 882 kB/s | 59 kB 00:00 2024-04-17T07:43:41.689 INFO:teuthology.orchestra.run.smithi053.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 761 kB/s | 51 kB 00:00 2024-04-17T07:43:41.714 INFO:teuthology.orchestra.run.smithi053.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.2 MB/s | 230 kB 00:00 2024-04-17T07:43:41.739 INFO:teuthology.orchestra.run.smithi053.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-17T07:43:41.764 INFO:teuthology.orchestra.run.smithi053.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-17T07:43:41.790 INFO:teuthology.orchestra.run.smithi053.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 404 kB/s | 20 kB 00:00 2024-04-17T07:43:41.807 INFO:teuthology.orchestra.run.smithi046.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-17T07:43:41.815 INFO:teuthology.orchestra.run.smithi053.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-17T07:43:41.857 INFO:teuthology.orchestra.run.smithi053.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-17T07:43:41.940 INFO:teuthology.orchestra.run.smithi053.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-04-17T07:43:42.007 INFO:teuthology.orchestra.run.smithi053.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-17T07:43:42.015 INFO:teuthology.orchestra.run.smithi046.stdout:(114/174): protobuf-compiler-3.14.0-13.el9.x86_ 448 kB/s | 863 kB 00:01 2024-04-17T07:43:42.041 INFO:teuthology.orchestra.run.smithi053.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-17T07:43:42.049 INFO:teuthology.orchestra.run.smithi046.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-17T07:43:42.074 INFO:teuthology.orchestra.run.smithi053.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 7.3 MB/s | 248 kB 00:00 2024-04-17T07:43:42.116 INFO:teuthology.orchestra.run.smithi053.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-17T07:43:42.157 INFO:teuthology.orchestra.run.smithi046.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 861 kB/s | 32 kB 00:00 2024-04-17T07:43:42.191 INFO:teuthology.orchestra.run.smithi046.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 466 kB/s | 15 kB 00:00 2024-04-17T07:43:42.208 INFO:teuthology.orchestra.run.smithi053.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 779 kB/s | 71 kB 00:00 2024-04-17T07:43:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:(118/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 201 kB/s | 43 kB 00:00 2024-04-17T07:43:42.258 INFO:teuthology.orchestra.run.smithi053.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 730 kB/s | 36 kB 00:00 2024-04-17T07:43:42.300 INFO:teuthology.orchestra.run.smithi046.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-04-17T07:43:42.325 INFO:teuthology.orchestra.run.smithi053.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-04-17T07:43:42.376 INFO:teuthology.orchestra.run.smithi053.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-04-17T07:43:42.400 INFO:teuthology.orchestra.run.smithi046.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-04-17T07:43:42.434 INFO:teuthology.orchestra.run.smithi053.stdout:(160/174): librados2-19.0.0-3133.ga8c84f59.el9. 5.5 MB/s | 3.4 MB 00:00 2024-04-17T07:43:42.442 INFO:teuthology.orchestra.run.smithi046.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 873 kB/s | 35 kB 00:00 2024-04-17T07:43:42.460 INFO:teuthology.orchestra.run.smithi053.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 275 kB/s | 23 kB 00:00 2024-04-17T07:43:42.476 INFO:teuthology.orchestra.run.smithi046.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-04-17T07:43:42.510 INFO:teuthology.orchestra.run.smithi053.stdout:(162/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-17T07:43:42.534 INFO:teuthology.orchestra.run.smithi046.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-04-17T07:43:42.535 INFO:teuthology.orchestra.run.smithi053.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-04-17T07:43:42.552 INFO:teuthology.orchestra.run.smithi053.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 926 kB/s | 108 kB 00:00 2024-04-17T07:43:42.586 INFO:teuthology.orchestra.run.smithi053.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-17T07:43:42.611 INFO:teuthology.orchestra.run.smithi053.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 489 kB/s | 29 kB 00:00 2024-04-17T07:43:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-17T07:43:42.644 INFO:teuthology.orchestra.run.smithi053.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi046.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 219 kB/s | 11 kB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi046.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 436 kB/s | 18 kB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi046.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 573 kB/s | 23 kB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi046.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 479 kB/s | 19 kB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi046.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 485 kB/s | 20 kB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.8 MB/s | 465 kB 00:00 2024-04-17T07:43:42.924 INFO:teuthology.orchestra.run.smithi053.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 536 kB/s | 31 kB 00:00 2024-04-17T07:43:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.5 MB/s | 2.2 MB 00:00 2024-04-17T07:43:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.4 MB/s | 2.3 MB 00:00 2024-04-17T07:43:42.929 INFO:teuthology.orchestra.run.smithi053.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.0 MB/s | 1.1 MB 00:00 2024-04-17T07:43:42.953 INFO:teuthology.orchestra.run.smithi046.stdout:(130/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-04-17T07:43:42.954 INFO:teuthology.orchestra.run.smithi053.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 636 kB/s | 74 kB 00:00 2024-04-17T07:43:42.958 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-04-17T07:43:42.958 INFO:teuthology.orchestra.run.smithi053.stdout:Total 15 MB/s | 229 MB 00:15 2024-04-17T07:43:42.995 INFO:teuthology.orchestra.run.smithi046.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-17T07:43:43.054 INFO:teuthology.orchestra.run.smithi046.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-17T07:43:43.096 INFO:teuthology.orchestra.run.smithi046.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-04-17T07:43:43.212 INFO:teuthology.orchestra.run.smithi046.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-17T07:43:43.272 INFO:teuthology.orchestra.run.smithi046.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-17T07:43:43.326 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-04-17T07:43:43.348 INFO:teuthology.orchestra.run.smithi046.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 2.0 MB/s | 1.0 MB 00:00 2024-04-17T07:43:43.422 INFO:teuthology.orchestra.run.smithi046.stdout:(137/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.2 MB/s | 2.0 MB 00:00 2024-04-17T07:43:43.447 INFO:teuthology.orchestra.run.smithi046.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 518 kB/s | 90 kB 00:00 2024-04-17T07:43:43.472 INFO:teuthology.orchestra.run.smithi046.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 229 kB/s | 28 kB 00:00 2024-04-17T07:43:43.498 INFO:teuthology.orchestra.run.smithi046.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 627 kB/s | 31 kB 00:00 2024-04-17T07:43:43.531 INFO:teuthology.orchestra.run.smithi046.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-17T07:43:43.573 INFO:teuthology.orchestra.run.smithi046.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 482 kB/s | 36 kB 00:00 2024-04-17T07:43:43.598 INFO:teuthology.orchestra.run.smithi046.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-17T07:43:43.622 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-04-17T07:43:43.622 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-04-17T07:43:43.631 INFO:teuthology.orchestra.run.smithi046.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 513 kB/s | 51 kB 00:00 2024-04-17T07:43:43.690 INFO:teuthology.orchestra.run.smithi046.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 989 kB/s | 90 kB 00:00 2024-04-17T07:43:43.732 INFO:teuthology.orchestra.run.smithi046.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 482 kB/s | 20 kB 00:00 2024-04-17T07:43:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-17T07:43:43.794 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:43:43.834 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:43:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-17T07:43:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-17T07:43:44.367 INFO:teuthology.orchestra.run.smithi046.stdout:(150/174): librados2-19.0.0-3133.ga8c84f59.el9. 6.7 MB/s | 3.4 MB 00:00 2024-04-17T07:43:44.403 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:43:44.438 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T07:43:44.484 INFO:teuthology.orchestra.run.smithi046.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 20 MB/s | 2.3 MB 00:00 2024-04-17T07:43:44.511 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T07:43:44.511 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T07:43:44.512 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T07:43:44.512 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:44.539 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T07:43:44.559 INFO:teuthology.orchestra.run.smithi046.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 25 MB/s | 1.8 MB 00:00 2024-04-17T07:43:44.584 INFO:teuthology.orchestra.run.smithi046.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.7 MB/s | 41 kB 00:00 2024-04-17T07:43:44.626 INFO:teuthology.orchestra.run.smithi046.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.9 MB/s | 248 kB 00:00 2024-04-17T07:43:44.668 INFO:teuthology.orchestra.run.smithi046.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.8 MB/s | 321 kB 00:00 2024-04-17T07:43:44.752 INFO:teuthology.orchestra.run.smithi046.stdout:(156/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-17T07:43:44.777 INFO:teuthology.orchestra.run.smithi046.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 664 kB/s | 71 kB 00:00 2024-04-17T07:43:44.995 INFO:teuthology.orchestra.run.smithi046.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 151 kB/s | 36 kB 00:00 2024-04-17T07:43:45.028 INFO:teuthology.orchestra.run.smithi046.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 769 kB/s | 192 kB 00:00 2024-04-17T07:43:45.078 INFO:teuthology.orchestra.run.smithi046.stdout:(160/174): thrift-0.15.0-2.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-04-17T07:43:45.103 INFO:teuthology.orchestra.run.smithi046.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 310 kB/s | 23 kB 00:00 2024-04-17T07:43:45.137 INFO:teuthology.orchestra.run.smithi046.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-04-17T07:43:45.170 INFO:teuthology.orchestra.run.smithi046.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-04-17T07:43:45.195 INFO:teuthology.orchestra.run.smithi046.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 926 kB/s | 108 kB 00:00 2024-04-17T07:43:45.221 INFO:teuthology.orchestra.run.smithi046.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-04-17T07:43:45.246 INFO:teuthology.orchestra.run.smithi046.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 581 kB/s | 29 kB 00:00 2024-04-17T07:43:45.271 INFO:teuthology.orchestra.run.smithi046.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-04-17T07:43:45.465 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-04-17T07:43:45.465 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-04-17T07:43:45.488 INFO:teuthology.orchestra.run.smithi046.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.5 MB/s | 1.2 MB 00:00 2024-04-17T07:43:45.563 INFO:teuthology.orchestra.run.smithi046.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.8 MB/s | 2.2 MB 00:00 2024-04-17T07:43:45.622 INFO:teuthology.orchestra.run.smithi046.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-04-17T07:43:45.647 INFO:teuthology.orchestra.run.smithi046.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 375 kB/s | 31 kB 00:00 2024-04-17T07:43:45.723 INFO:teuthology.orchestra.run.smithi046.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.1 MB/s | 2.3 MB 00:00 2024-04-17T07:43:45.748 INFO:teuthology.orchestra.run.smithi046.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 753 kB/s | 74 kB 00:00 2024-04-17T07:43:45.798 INFO:teuthology.orchestra.run.smithi046.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-04-17T07:43:45.814 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-04-17T07:43:45.814 INFO:teuthology.orchestra.run.smithi046.stdout:Total 12 MB/s | 229 MB 00:19 2024-04-17T07:43:46.980 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-04-17T07:43:47.385 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-04-17T07:43:47.385 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-04-17T07:43:49.868 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-04-17T07:43:49.868 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-04-17T07:43:52.667 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-04-17T07:43:52.759 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T07:43:52.810 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T07:43:52.840 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T07:43:52.849 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T07:43:52.899 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T07:43:52.953 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T07:43:52.986 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T07:43:53.011 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T07:43:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T07:43:53.401 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T07:43:53.453 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:43:53.561 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:43:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:43:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:43:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T07:43:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T07:43:53.943 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T07:43:53.998 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T07:43:54.026 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:43:54.131 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:43:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T07:43:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T07:43:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T07:43:54.278 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T07:43:54.294 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T07:43:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T07:43:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T07:43:54.605 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T07:43:54.716 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T07:43:54.801 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T07:43:54.845 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T07:43:54.915 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:43:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:43:55.194 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:43:55.306 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T07:43:55.402 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T07:43:55.434 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T07:43:55.461 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T07:43:55.497 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T07:43:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T07:43:55.626 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T07:43:55.663 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T07:43:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T07:43:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T07:43:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T07:43:55.854 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T07:43:55.882 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T07:43:55.963 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T07:43:55.995 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T07:43:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T07:43:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T07:43:56.133 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T07:43:56.183 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T07:43:56.260 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T07:43:56.405 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T07:43:56.462 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T07:43:56.816 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-04-17T07:43:56.817 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:57.002 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T07:43:57.015 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T07:43:57.018 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T07:43:57.018 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:43:57.018 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T07:43:57.019 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:57.099 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T07:43:57.191 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T07:43:57.249 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T07:43:57.258 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T07:43:57.289 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T07:43:57.339 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T07:43:57.376 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T07:43:57.405 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T07:43:57.611 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-04-17T07:43:57.640 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T07:43:57.730 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T07:43:57.779 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T07:43:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T07:43:57.834 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T07:43:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T07:43:57.923 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T07:43:57.951 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T07:43:57.967 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T07:43:58.058 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T07:43:58.106 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T07:43:58.108 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T07:43:58.156 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T07:43:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T07:43:58.253 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T07:43:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T07:43:58.501 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T07:43:58.537 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T07:43:58.555 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T07:43:58.574 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T07:43:58.582 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T07:43:58.582 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:43:58.582 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T07:43:58.582 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T07:43:58.582 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T07:43:58.583 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:58.621 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T07:43:58.656 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T07:43:58.677 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T07:43:58.700 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T07:43:58.730 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:43:58.742 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T07:43:58.823 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T07:43:58.847 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T07:43:58.847 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:43:58.847 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T07:43:58.847 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T07:43:58.847 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T07:43:58.847 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:58.884 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T07:43:58.893 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:43:58.914 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:43:58.938 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T07:43:58.968 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T07:43:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T07:43:59.038 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:43:59.063 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T07:43:59.109 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T07:43:59.143 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T07:43:59.148 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T07:43:59.183 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T07:43:59.208 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T07:43:59.225 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T07:43:59.292 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T07:43:59.315 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T07:43:59.339 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T07:43:59.339 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:43:59.339 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T07:43:59.339 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T07:43:59.340 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T07:43:59.340 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:43:59.424 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T07:43:59.433 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T07:43:59.460 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T07:43:59.519 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T07:43:59.529 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T07:43:59.544 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:43:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T07:43:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T07:43:59.690 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T07:43:59.713 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:43:59.753 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T07:43:59.753 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T07:43:59.782 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T07:43:59.814 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T07:43:59.854 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T07:43:59.877 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T07:43:59.884 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T07:43:59.904 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T07:43:59.907 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T07:43:59.930 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T07:43:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T07:44:00.002 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T07:44:00.077 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T07:44:00.111 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T07:44:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T07:44:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T07:44:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T07:44:00.235 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T07:44:00.275 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T07:44:00.314 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T07:44:00.315 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T07:44:00.344 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T07:44:00.349 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T07:44:00.389 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T07:44:00.424 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T07:44:00.463 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T07:44:00.496 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T07:44:00.538 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T07:44:00.572 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T07:44:00.667 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T07:44:00.725 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T07:44:00.755 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T07:44:00.920 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T07:44:00.978 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T07:44:01.001 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T07:44:01.009 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T07:44:01.040 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T07:44:01.168 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T07:44:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:44:01.217 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T07:44:01.257 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:44:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T07:44:01.387 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T07:44:01.603 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T07:44:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T07:44:01.893 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T07:44:01.930 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T07:44:01.966 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T07:44:02.026 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T07:44:02.056 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T07:44:02.082 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T07:44:02.150 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T07:44:02.159 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T07:44:02.190 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T07:44:02.207 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T07:44:02.214 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T07:44:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T07:44:02.253 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T07:44:02.308 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T07:44:02.348 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T07:44:02.361 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T07:44:02.392 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T07:44:02.393 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T07:44:02.400 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T07:44:02.417 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T07:44:02.417 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:02.417 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T07:44:02.417 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T07:44:02.417 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T07:44:02.417 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:44:02.433 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T07:44:02.456 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T07:44:02.476 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T07:44:02.487 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T07:44:02.500 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T07:44:02.533 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T07:44:02.548 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T07:44:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T07:44:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T07:44:02.661 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T07:44:02.703 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T07:44:02.729 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T07:44:02.770 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T07:44:02.802 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T07:44:02.917 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T07:44:02.970 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T07:44:03.096 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T07:44:03.320 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T07:44:03.391 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T07:44:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T07:44:03.511 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T07:44:03.536 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T07:44:03.542 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T07:44:03.560 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T07:44:03.561 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:03.561 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T07:44:03.561 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T07:44:03.561 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T07:44:03.561 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:44:03.604 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T07:44:03.627 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T07:44:03.627 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:03.627 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T07:44:03.627 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:44:03.680 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:03.704 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:03.704 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:03.704 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T07:44:03.704 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T07:44:03.704 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T07:44:03.704 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:44:04.030 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T07:44:04.054 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T07:44:04.054 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:04.054 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T07:44:04.054 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T07:44:04.054 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T07:44:04.054 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:44:04.124 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T07:44:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T07:44:04.341 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T07:44:04.396 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T07:44:04.424 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T07:44:04.470 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T07:44:04.562 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T07:44:04.617 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T07:44:04.663 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T07:44:04.843 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T07:44:04.931 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T07:44:04.961 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T07:44:04.992 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T07:44:05.020 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T07:44:05.094 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T07:44:05.287 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T07:44:05.349 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T07:44:05.383 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T07:44:05.401 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T07:44:05.449 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T07:44:05.458 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T07:44:05.488 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T07:44:05.525 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T07:44:05.683 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T07:44:05.731 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T07:44:05.813 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T07:44:05.912 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T07:44:05.969 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T07:44:06.021 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T07:44:06.031 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T07:44:06.061 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T07:44:06.079 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T07:44:06.119 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T07:44:06.149 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T07:44:06.171 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T07:44:06.181 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T07:44:06.242 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T07:44:06.439 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T07:44:06.501 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T07:44:06.539 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T07:44:06.599 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T07:44:06.742 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T07:44:06.787 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T07:44:06.834 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T07:44:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T07:44:06.894 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T07:44:07.080 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T07:44:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T07:44:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T07:44:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T07:44:07.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T07:44:07.620 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T07:44:07.873 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T07:44:07.957 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T07:44:08.145 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T07:44:08.200 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T07:44:08.241 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T07:44:08.279 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T07:44:08.320 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T07:44:08.367 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T07:44:08.433 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T07:44:08.494 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T07:44:08.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T07:44:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:44:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:44:08.674 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T07:44:08.819 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T07:44:08.910 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T07:44:08.952 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T07:44:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T07:44:08.981 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T07:44:09.008 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T07:44:09.151 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:44:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T07:44:09.202 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T07:44:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T07:44:09.253 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T07:44:09.253 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T07:44:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T07:44:09.322 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T07:44:09.362 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T07:44:09.367 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T07:44:09.406 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T07:44:09.422 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T07:44:09.458 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T07:44:09.502 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T07:44:09.534 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T07:44:09.566 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T07:44:09.589 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T07:44:09.596 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T07:44:09.651 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T07:44:09.716 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T07:44:09.734 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T07:44:09.753 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T07:44:09.796 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T07:44:09.832 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T07:44:09.890 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T07:44:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T07:44:10.099 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T07:44:10.323 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:10.367 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:10.388 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:11.128 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T07:44:11.180 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T07:44:11.213 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T07:44:11.271 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T07:44:11.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T07:44:11.388 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T07:44:11.404 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T07:44:11.422 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T07:44:11.445 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T07:44:11.528 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T07:44:11.557 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T07:44:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T07:44:11.608 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T07:44:11.634 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T07:44:11.635 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T07:44:11.650 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T07:44:11.685 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T07:44:11.706 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T07:44:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T07:44:11.735 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T07:44:11.791 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T07:44:11.803 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T07:44:11.830 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T07:44:11.862 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T07:44:11.888 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T07:44:11.913 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T07:44:11.946 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T07:44:11.986 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T07:44:12.018 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T07:44:12.043 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T07:44:12.068 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T07:44:12.093 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T07:44:12.119 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T07:44:12.126 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:12.126 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:12.313 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:12.314 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T07:44:12.388 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T07:44:12.420 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T07:44:12.446 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T07:44:12.472 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T07:44:12.505 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T07:44:12.589 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T07:44:12.661 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T07:44:12.805 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T07:44:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T07:44:14.523 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T07:44:14.578 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T07:44:14.608 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T07:44:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T07:44:15.021 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T07:44:15.072 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T07:44:15.194 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T07:44:15.228 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T07:44:15.285 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T07:44:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T07:44:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T07:44:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T07:44:16.043 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T07:44:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T07:44:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T07:44:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T07:44:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T07:44:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T07:44:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T07:44:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T07:44:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T07:44:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T07:44:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T07:44:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T07:44:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T07:44:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T07:44:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T07:44:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T07:44:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T07:44:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T07:44:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T07:44:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T07:44:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T07:44:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T07:44:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T07:44:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T07:44:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T07:44:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T07:44:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T07:44:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T07:44:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T07:44:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T07:44:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T07:44:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T07:44:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T07:44:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T07:44:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T07:44:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T07:44:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T07:44:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T07:44:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T07:44:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T07:44:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T07:44:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T07:44:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T07:44:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T07:44:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T07:44:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T07:44:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T07:44:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T07:44:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T07:44:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T07:44:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T07:44:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T07:44:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T07:44:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T07:44:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T07:44:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T07:44:18.881 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:44:18.925 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:44:19.585 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T07:44:19.628 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T07:44:19.792 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T07:44:19.792 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T07:44:19.792 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T07:44:19.792 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:19.837 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T07:44:21.271 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T07:44:21.271 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:44:21.271 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-04-17T07:44:21.271 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:21.271 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:21.271 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:21.271 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.272 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T07:44:21.275 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-16.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T07:44:21.276 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T07:44:21.277 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T07:44:21.278 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T07:44:21.279 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T07:44:21.280 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T07:44:21.281 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T07:44:21.282 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:21.283 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T07:44:21.284 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T07:44:21.284 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T07:44:21.284 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T07:44:21.284 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T07:44:21.284 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T07:44:21.284 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T07:44:21.285 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:44:21.285 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-17T07:44:21.512 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T07:44:21.590 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-04-17T07:44:21.591 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-04-17T07:44:21.591 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-04-17T07:44:21.591 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-04-17T07:44:21.591 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-04-17T07:44:21.591 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-04-17T07:44:21.591 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-04-17T07:44:21.591 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:21.685 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T07:44:21.699 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T07:44:21.700 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:21.700 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T07:44:21.700 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:21.738 DEBUG:teuthology.parallel:result is None 2024-04-17T07:44:23.236 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T07:44:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T07:44:23.263 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:23.263 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T07:44:23.263 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T07:44:23.263 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T07:44:23.263 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:23.495 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T07:44:23.519 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T07:44:23.519 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:23.519 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T07:44:23.519 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T07:44:23.519 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T07:44:23.519 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:23.987 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T07:44:24.011 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T07:44:24.011 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:24.011 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T07:44:24.011 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T07:44:24.011 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T07:44:24.011 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:24.106 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T07:44:24.535 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T07:44:24.562 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T07:44:24.595 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T07:44:24.658 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T07:44:24.685 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T07:44:25.787 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T07:44:25.820 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T07:44:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T07:44:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T07:44:26.968 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T07:44:27.141 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T07:44:27.172 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T07:44:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T07:44:27.197 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:27.197 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T07:44:27.197 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T07:44:27.197 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T07:44:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T07:44:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T07:44:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T07:44:28.283 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T07:44:28.319 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T07:44:28.342 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T07:44:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T07:44:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T07:44:28.343 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T07:44:28.343 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T07:44:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T07:44:28.419 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:28.419 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T07:44:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:28.470 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:28.493 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:28.494 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:28.494 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T07:44:28.494 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T07:44:28.494 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T07:44:28.494 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:28.812 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T07:44:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T07:44:28.836 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:28.836 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T07:44:28.836 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T07:44:28.836 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T07:44:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:33.320 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T07:44:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-04-17T07:44:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-04-17T07:44:33.321 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-04-17T07:44:33.321 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-04-17T07:44:33.321 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-04-17T07:44:33.321 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-04-17T07:44:33.321 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-04-17T07:44:33.321 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:33.543 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T07:44:33.594 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T07:44:33.595 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:33.595 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T07:44:33.595 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T07:44:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T07:44:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T07:44:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T07:44:33.997 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T07:44:34.111 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T07:44:34.157 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T07:44:34.194 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T07:44:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T07:44:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T07:44:34.314 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T07:44:34.354 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T07:44:34.376 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T07:44:34.430 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T07:44:34.491 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T07:44:34.524 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T07:44:35.065 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:35.104 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:35.125 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:35.270 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T07:44:35.351 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T07:44:35.351 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:35.351 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T07:44:35.351 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T07:44:35.351 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T07:44:35.352 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:35.611 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T07:44:35.684 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T07:44:35.685 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:35.685 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T07:44:35.685 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T07:44:35.685 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T07:44:35.685 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:36.012 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T07:44:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T07:44:36.141 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T07:44:36.195 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T07:44:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T07:44:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T07:44:36.265 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T07:44:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T07:44:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T07:44:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T07:44:36.266 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T07:44:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T07:44:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T07:44:36.330 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T07:44:36.389 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T07:44:36.418 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T07:44:36.447 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T07:44:36.447 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T07:44:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T07:44:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T07:44:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T07:44:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T07:44:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T07:44:36.675 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T07:44:36.722 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T07:44:36.762 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T07:44:36.788 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T07:44:36.833 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T07:44:36.863 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T07:44:36.888 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T07:44:36.895 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:36.895 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:37.003 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T07:44:37.048 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T07:44:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T07:44:37.101 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T07:44:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T07:44:37.197 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T07:44:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T07:44:37.227 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T07:44:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T07:44:37.258 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T07:44:37.283 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T07:44:37.350 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T07:44:38.580 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T07:44:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T07:44:39.781 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T07:44:39.804 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T07:44:39.953 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T07:44:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T07:44:40.189 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T07:44:40.289 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T07:44:40.289 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:40.289 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T07:44:40.289 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T07:44:40.289 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T07:44:40.289 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:40.357 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T07:44:40.397 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T07:44:40.528 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T07:44:41.611 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T07:44:41.651 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T07:44:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T07:44:41.751 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:41.751 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T07:44:41.751 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T07:44:41.751 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T07:44:41.751 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:41.801 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T07:44:41.905 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T07:44:41.906 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:41.906 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T07:44:41.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:41.994 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:42.103 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:42.104 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:42.104 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T07:44:42.104 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T07:44:42.104 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T07:44:42.104 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:42.511 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T07:44:42.604 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T07:44:42.604 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T07:44:42.604 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T07:44:42.605 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T07:44:42.605 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T07:44:42.605 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T07:44:42.740 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T07:44:42.741 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T07:44:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T07:44:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T07:44:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T07:44:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T07:44:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T07:44:42.742 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T07:44:42.758 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T07:44:42.758 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T07:44:42.759 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T07:44:42.760 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T07:44:42.761 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T07:44:42.762 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T07:44:42.763 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T07:44:42.764 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T07:44:42.765 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T07:44:42.766 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T07:44:42.767 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T07:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T07:44:42.769 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T07:44:42.770 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T07:44:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T07:44:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T07:44:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T07:44:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T07:44:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T07:44:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T07:44:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T07:44:46.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T07:44:46.036 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:46.036 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.037 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T07:44:46.038 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.039 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:44:46.040 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-16.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.041 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T07:44:46.042 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T07:44:46.043 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T07:44:46.044 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T07:44:46.045 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T07:44:46.046 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T07:44:46.047 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T07:44:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T07:44:46.049 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T07:44:46.049 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T07:44:46.049 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T07:44:46.049 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-17T07:44:46.049 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-04-17T07:44:46.507 DEBUG:teuthology.parallel:result is None 2024-04-17T07:44:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T07:44:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T07:44:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T07:44:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T07:44:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T07:44:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T07:44:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T07:44:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T07:44:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T07:44:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T07:44:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T07:44:48.773 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T07:44:48.837 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T07:44:48.931 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T07:44:49.041 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T07:44:49.084 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T07:44:49.988 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:50.031 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:50.096 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T07:44:51.577 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T07:44:51.678 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T07:44:51.748 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T07:44:51.902 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T07:44:51.941 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T07:44:51.974 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T07:44:52.009 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T07:44:52.051 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T07:44:52.051 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T07:44:52.216 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T07:44:52.271 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T07:44:52.271 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T07:44:52.409 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T07:44:52.454 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T07:44:52.499 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T07:44:52.532 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T07:44:52.573 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T07:44:52.646 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T07:44:52.730 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T07:44:52.778 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T07:44:52.811 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T07:44:52.845 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T07:44:52.878 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T07:44:52.911 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T07:44:52.950 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:52.950 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:53.295 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T07:44:53.296 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T07:44:53.435 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T07:44:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T07:44:53.523 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T07:44:53.560 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T07:44:53.600 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T07:44:53.722 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T07:44:53.994 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T07:44:53.994 INFO:teuthology.orchestra.run.smithi046.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-17T07:44:53.994 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:44:59.256 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T07:44:59.256 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:44:59.257 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T07:44:59.258 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T07:44:59.259 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T07:44:59.260 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T07:44:59.260 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T07:44:59.260 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T07:44:59.260 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T07:44:59.279 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T07:44:59.279 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T07:44:59.279 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T07:44:59.279 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T07:44:59.279 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T07:44:59.279 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T07:44:59.280 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T07:44:59.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T07:44:59.282 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T07:44:59.283 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T07:44:59.284 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T07:44:59.285 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T07:44:59.286 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T07:44:59.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T07:44:59.288 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T07:44:59.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T07:44:59.290 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T07:44:59.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T07:44:59.292 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T07:44:59.293 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T07:44:59.293 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T07:44:59.293 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T07:44:59.293 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T07:45:01.798 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T07:45:01.798 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:45:01.798 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.799 INFO:teuthology.orchestra.run.smithi046.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T07:45:01.800 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.801 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.802 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: jq-1.6-16.el9.x86_64 2024-04-17T07:45:01.803 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.804 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T07:45:01.805 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T07:45:01.806 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T07:45:01.807 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T07:45:01.808 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T07:45:01.809 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T07:45:01.810 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T07:45:01.811 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T07:45:01.812 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T07:45:01.813 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T07:45:01.813 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:45:01.813 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-04-17T07:45:02.691 DEBUG:teuthology.parallel:result is None 2024-04-17T07:45:02.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:45:02.927 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T07:45:02.989 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T07:45:02.990 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T07:45:02.990 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T07:45:02.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:45:03.218 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T07:45:03.245 INFO:teuthology.orchestra.run.smithi053.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T07:45:03.245 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T07:45:03.246 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T07:45:03.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:45:03.513 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T07:45:03.540 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T07:45:03.541 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T07:45:03.541 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T07:45:03.543 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T07:45:03.544 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:45:03.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T07:45:03.626 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:45:03.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T07:45:03.667 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:45:03.668 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T07:45:03.707 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T07:45:03.708 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:45:03.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T07:45:03.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T07:45:03.902 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:45:03.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T07:45:03.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T07:45:04.015 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:45:04.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T07:45:04.053 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T07:45:04.126 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T07:45:04.127 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:45:04.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T07:45:04.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T07:45:04.327 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:45:04.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T07:45:04.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T07:45:04.470 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:45:04.471 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T07:45:04.502 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T07:45:04.574 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T07:45:04.574 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:45:04.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T07:45:04.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T07:45:04.781 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:45:04.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T07:45:04.812 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T07:45:04.884 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:45:04.884 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T07:45:04.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T07:45:04.987 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-17T07:45:05.212 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-17T07:45:05.213 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:45:05.213 INFO:tasks.cephadm:Cluster fsid is 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:45:05.213 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-17T07:45:05.213 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.b': '172.21.15.53', 'mon.c': '172.21.15.62'} 2024-04-17T07:45:05.214 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-04-17T07:45:05.214 INFO:tasks.cephadm:First mgr is x 2024-04-17T07:45:05.214 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-17T07:45:05.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-04-17T07:45:05.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-04-17T07:45:05.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-04-17T07:45:05.363 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-17T07:45:05.363 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:45:05.502 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-17T07:45:05.787 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735, sha1 a8c84f5997950fd7d008204839253291d52e6eda from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:45:05.788 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm 2024-04-17T07:45:05.789 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:06.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 07:45 /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:06.835 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:07.375 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 07:45 /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:07.375 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:08.568 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 07:45 /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:08.569 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:08.634 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:08.653 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T07:45:08.693 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda on all hosts... 2024-04-17T07:45:08.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T07:45:08.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T07:45:08.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T07:45:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:45:09.510 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:45:09.919 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:45:37.086 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-17T07:45:37.087 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:45:37.087 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T07:45:37.087 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-04-17T07:45:37.087 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T07:45:37.087 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-17T07:45:37.087 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-17T07:45:46.048 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-17T07:45:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:45:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T07:45:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-04-17T07:45:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T07:45:46.048 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-04-17T07:45:46.049 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-17T07:45:50.132 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:45:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:45:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T07:45:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-04-17T07:45:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T07:45:50.133 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T07:45:50.133 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:45:50.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-04-17T07:45:50.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-04-17T07:45:50.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-04-17T07:45:50.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-04-17T07:45:50.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-04-17T07:45:50.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-04-17T07:45:50.562 INFO:tasks.cephadm:Writing seed config... 2024-04-17T07:45:50.563 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-17T07:45:50.563 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-17T07:45:50.563 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-17T07:45:50.563 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-17T07:45:50.564 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-17T07:45:50.565 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-17T07:45:50.566 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-17T07:45:50.567 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:45:50.567 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-17T07:45:50.628 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 67cd6c06-fc8e-11ee-bc91-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-17T07:45:50.629 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a.service 2024-04-17T07:45:50.676 DEBUG:teuthology.orchestra.run.smithi046:mgr.x> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.x.service 2024-04-17T07:45:50.723 INFO:tasks.cephadm:Bootstrapping... 2024-04-17T07:45:50.723 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda -v bootstrap --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:45:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-04-17T07:45:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda', '-v', 'bootstrap', '--fsid', '67cd6c06-fc8e-11ee-bc91-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-04-17T07:45:51.010 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-17T07:45:51.010 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-04-17T07:45:51.073 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-17T07:45:51.073 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-04-17T07:45:51.073 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-04-17T07:45:51.090 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-17T07:45:51.090 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-17T07:45:51.106 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-17T07:45:51.106 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-04-17T07:45:51.129 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-04-17T07:45:51.152 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-04-17T07:45:51.152 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-04-17T07:45:51.152 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-04-17T07:45:51.213 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-17T07:45:51.213 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-17T07:45:51.213 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-04-17T07:45:51.213 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-04-17T07:45:51.231 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-17T07:45:51.231 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-17T07:45:51.250 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-17T07:45:51.251 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-04-17T07:45:51.273 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-04-17T07:45:51.296 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-04-17T07:45:51.296 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-04-17T07:45:51.296 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-04-17T07:45:51.296 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:45:51.297 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 139893259953056 on /run/cephadm/67cd6c06-fc8e-11ee-bc91-c7b262605968.lock 2024-04-17T07:45:51.297 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 139893259953056 acquired on /run/cephadm/67cd6c06-fc8e-11ee-bc91-c7b262605968.lock 2024-04-17T07:45:51.299 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-04-17T07:45:51.300 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-04-17T07:45:51.300 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-04-17T07:45:51.315 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.46 metric 100 2024-04-17T07:45:51.316 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-04-17T07:45:51.329 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-17T07:45:51.329 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-17T07:45:51.329 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-17T07:45:51.341 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-17T07:45:51.341 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-17T07:45:51.341 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-17T07:45:51.342 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-17T07:45:51.342 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-17T07:45:51.342 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-17T07:45:51.344 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-04-17T07:45:51.344 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-04-17T07:45:51.344 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-17T07:45:51.347 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-17T07:45:51.350 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:45:52.056 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-17T07:45:52.056 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:45:52.056 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-04-17T07:45:52.056 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:9473d3fd79377e8e2904ab0f292b04da81cb2f2bfccde15ec927b481a725f6f4 2024-04-17T07:45:52.057 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-17T07:45:52.057 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-17T07:45:52.057 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-17T07:45:53.322 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T07:45:53.323 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T07:45:53.323 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-04-17T07:46:03.203 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-04-17T07:46:03.204 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-04-17T07:46:04.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQA7fh9mt8k6KRAANV+O5/xM4IUtXjNnJzY7ZA== 2024-04-17T07:46:05.618 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQA8fh9mUeQPNBAApBbBCm3/S7RgwMX4py3Kgg== 2024-04-17T07:46:06.771 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQA+fh9m06aJCBAAjXqswqUvOLpSZ1gr7gstnA== 2024-04-17T07:46:06.773 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-04-17T07:46:08.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-17T07:46:08.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:46:08.385 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-04-17T07:46:09.618 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-04-17T07:46:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-17T07:46:10.285 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-17T07:46:10.589 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968.target → /etc/systemd/system/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968.target. 2024-04-17T07:46:10.590 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968.target → /etc/systemd/system/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968.target. 2024-04-17T07:46:10.957 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a 2024-04-17T07:46:10.957 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a.service: Unit ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a.service not loaded. 2024-04-17T07:46:11.245 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968.target.wants/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a.service → /etc/systemd/system/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@.service. 2024-04-17T07:46:11.982 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:11 smithi046 podman[31658]: 2024-04-17 07:46:11.8965071 +0000 UTC m=+0.380889336 container start 241e8f68d1c535b884d025c50ad9d76ab812d9f617188e554e714df028726fa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:46:12.044 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:46:12.044 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-04-17T07:46:12.064 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:46:12.064 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-04-17T07:46:12.064 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-04-17T07:46:12.064 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-04-17T07:46:12.066 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-04-17T07:46:12.066 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-04-17T07:46:12.349 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:11 smithi046 ceph-mon[31677]: mkfs 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:12.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:11 smithi046 ceph-mon[31677]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T07:46:12.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 bash[31658]: 241e8f68d1c535b884d025c50ad9d76ab812d9f617188e554e714df028726fa6 2024-04-17T07:46:12.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 systemd[1]: Started Ceph mon.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:46:13.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T07:46:13.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: monmap epoch 1 2024-04-17T07:46:13.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:13.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: last_changed 2024-04-17T07:46:07.693966+0000 2024-04-17T07:46:13.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:46:13.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: min_mon_release 19 (squid) 2024-04-17T07:46:13.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: election_strategy: 1 2024-04-17T07:46:13.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:46:13.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: fsmap 2024-04-17T07:46:13.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: osdmap e1: 0 total, 0 up, 0 in 2024-04-17T07:46:13.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: mgrmap e1: no daemons active 2024-04-17T07:46:13.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:12 smithi046 ceph-mon[31677]: from='client.? 172.21.15.46:0/2957569845' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-17T07:46:13.486 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.809121s) 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-17T07:46:13.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.690+0000 7fab53e00700 1 Processor -- start 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.690+0000 7fab53e00700 1 -- start start 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.690+0000 7fab53e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.690+0000 7fab53e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c110890 con 0x7fab4c10fea0 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.692+0000 7fab51400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.692+0000 7fab51400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39426/0 (socket says 172.21.15.46:39426) 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.692+0000 7fab51400700 1 -- 172.21.15.46:0/1139072281 learned_addr learned my addr 172.21.15.46:0/1139072281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.692+0000 7fab51400700 1 -- 172.21.15.46:0/1139072281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c110a10 con 0x7fab4c10fea0 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.693+0000 7fab51400700 1 --2- 172.21.15.46:0/1139072281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1102c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab400098a0 tx=0x7fab40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e649499591da8411 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:13.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.693+0000 7fab4be00700 1 -- 172.21.15.46:0/1139072281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab40004160 con 0x7fab4c10fea0 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.693+0000 7fab4be00700 1 -- 172.21.15.46:0/1139072281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab40004300 con 0x7fab4c10fea0 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.693+0000 7fab4be00700 1 -- 172.21.15.46:0/1139072281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab400036a0 con 0x7fab4c10fea0 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.694+0000 7fab53e00700 1 -- 172.21.15.46:0/1139072281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 msgr2=0x7fab4c1102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.694+0000 7fab53e00700 1 --2- 172.21.15.46:0/1139072281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1102c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab400098a0 tx=0x7fab40009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.694+0000 7fab53e00700 1 -- 172.21.15.46:0/1139072281 shutdown_connections 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.694+0000 7fab53e00700 1 --2- 172.21.15.46:0/1139072281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1102c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.694+0000 7fab53e00700 1 -- 172.21.15.46:0/1139072281 >> 172.21.15.46:0/1139072281 conn(0x7fab4c07d9c0 msgr2=0x7fab4c07fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.694+0000 7fab53e00700 1 -- 172.21.15.46:0/1139072281 shutdown_connections 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.694+0000 7fab53e00700 1 -- 172.21.15.46:0/1139072281 wait complete. 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.695+0000 7fab53e00700 1 Processor -- start 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.695+0000 7fab53e00700 1 -- start start 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.696+0000 7fab53e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.696+0000 7fab53e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c10ef20 con 0x7fab4c10fea0 2024-04-17T07:46:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.696+0000 7fab51400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.696+0000 7fab51400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39432/0 (socket says 172.21.15.46:39432) 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.696+0000 7fab51400700 1 -- 172.21.15.46:0/2957569845 learned_addr learned my addr 172.21.15.46:0/2957569845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.696+0000 7fab51400700 1 -- 172.21.15.46:0/2957569845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c1a2560 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.697+0000 7fab51400700 1 --2- 172.21.15.46:0/2957569845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1a2020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab40000c00 tx=0x7fab40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.697+0000 7fab4a000700 1 -- 172.21.15.46:0/2957569845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab40003db0 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.697+0000 7fab4a000700 1 -- 172.21.15.46:0/2957569845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab40011630 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.697+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c1a27f0 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.697+0000 7fab4a000700 1 -- 172.21.15.46:0/2957569845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab40011950 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.697+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c1a54e0 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.697+0000 7fab4a000700 1 -- 172.21.15.46:0/2957569845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab4001a460 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.698+0000 7fab4a000700 1 -- 172.21.15.46:0/2957569845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab4001b780 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.698+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4c0640b0 con 0x7fab4c10fea0 2024-04-17T07:46:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.702+0000 7fab4a000700 1 -- 172.21.15.46:0/2957569845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fab4001b980 con 0x7fab4c10fea0 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.783+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fab4c02e1e0 con 0x7fab4c10fea0 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.784+0000 7fab4a000700 1 -- 172.21.15.46:0/2957569845 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fab4001bb60 con 0x7fab4c10fea0 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.784+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 msgr2=0x7fab4c1a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.784+0000 7fab53e00700 1 --2- 172.21.15.46:0/2957569845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1a2020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab40000c00 tx=0x7fab40004750 comp rx=0 tx=0).stop 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.785+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 shutdown_connections 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.785+0000 7fab53e00700 1 --2- 172.21.15.46:0/2957569845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab4c10fea0 0x7fab4c1a2020 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.785+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 >> 172.21.15.46:0/2957569845 conn(0x7fab4c07d9c0 msgr2=0x7fab4c198cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.785+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 shutdown_connections 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:12.785+0000 7fab53e00700 1 -- 172.21.15.46:0/2957569845 wait complete. 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-04-17T07:46:13.498 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-04-17T07:46:14.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:14 smithi046 ceph-mon[31677]: from='client.? 172.21.15.46:0/1145794477' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-17T07:46:14.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:14 smithi046 ceph-mon[31677]: from='client.? 172.21.15.46:0/1145794477' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:15.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.203+0000 7f20c6200700 1 Processor -- start 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.203+0000 7f20c6200700 1 -- start start 2024-04-17T07:46:15.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.204+0000 7f20c6200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c010b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.204+0000 7f20c6200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20c007bdc0 con 0x7f20c0108c60 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.204+0000 7f20c4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c010b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.204+0000 7f20c4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c010b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39434/0 (socket says 172.21.15.46:39434) 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.204+0000 7f20c4e00700 1 -- 172.21.15.46:0/2704155545 learned_addr learned my addr 172.21.15.46:0/2704155545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.205+0000 7f20c4e00700 1 -- 172.21.15.46:0/2704155545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c010b590 con 0x7f20c0108c60 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.205+0000 7f20c4e00700 1 --2- 172.21.15.46:0/2704155545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c010b050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20b40098a0 tx=0x7f20b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1e9b64e05eedd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.205+0000 7f20bf400700 1 -- 172.21.15.46:0/2704155545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b4004160 con 0x7f20c0108c60 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.205+0000 7f20bf400700 1 -- 172.21.15.46:0/2704155545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b4004300 con 0x7f20c0108c60 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.205+0000 7f20bf400700 1 -- 172.21.15.46:0/2704155545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b40036a0 con 0x7f20c0108c60 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.206+0000 7f20c6200700 1 -- 172.21.15.46:0/2704155545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 msgr2=0x7f20c010b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.206+0000 7f20c6200700 1 --2- 172.21.15.46:0/2704155545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c010b050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20b40098a0 tx=0x7f20b4009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:15.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.206+0000 7f20c6200700 1 -- 172.21.15.46:0/2704155545 shutdown_connections 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.206+0000 7f20c6200700 1 --2- 172.21.15.46:0/2704155545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c010b050 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.206+0000 7f20c6200700 1 -- 172.21.15.46:0/2704155545 >> 172.21.15.46:0/2704155545 conn(0x7f20c0102610 msgr2=0x7f20c0104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.206+0000 7f20c6200700 1 -- 172.21.15.46:0/2704155545 shutdown_connections 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.206+0000 7f20c6200700 1 -- 172.21.15.46:0/2704155545 wait complete. 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.207+0000 7f20c6200700 1 Processor -- start 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.208+0000 7f20c6200700 1 -- start start 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.208+0000 7f20c6200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.208+0000 7f20c6200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20c010b9a0 con 0x7f20c0108c60 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.208+0000 7f20c4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.208+0000 7f20c4e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c019db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39438/0 (socket says 172.21.15.46:39438) 2024-04-17T07:46:15.020 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.208+0000 7f20c4e00700 1 -- 172.21.15.46:0/1145794477 learned_addr learned my addr 172.21.15.46:0/1145794477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.209+0000 7f20c4e00700 1 -- 172.21.15.46:0/1145794477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c019e050 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.209+0000 7f20c4e00700 1 --2- 172.21.15.46:0/1145794477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c019db10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f20b4000c00 tx=0x7f20b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.209+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b4003db0 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.209+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b4011630 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.209+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b4011950 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.209+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c019e2e0 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.209+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c01a0fd0 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.210+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20b401a460 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.210+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f20b401b780 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.211+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c00640b0 con 0x7f20c0108c60 2024-04-17T07:46:15.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.214+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f20b401ba20 con 0x7f20c0108c60 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.293+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f20c002e1e0 con 0x7f20c0108c60 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.302+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f20b401bc00 con 0x7f20c0108c60 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.303+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 msgr2=0x7f20c019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.304+0000 7f20c6200700 1 --2- 172.21.15.46:0/1145794477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c019db10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f20b4000c00 tx=0x7f20b4004750 comp rx=0 tx=0).stop 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.304+0000 7f20bd600700 1 -- 172.21.15.46:0/1145794477 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20b401bde0 con 0x7f20c0108c60 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.304+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 shutdown_connections 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.304+0000 7f20c6200700 1 --2- 172.21.15.46:0/1145794477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20c0108c60 0x7f20c019db10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.304+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 >> 172.21.15.46:0/1145794477 conn(0x7f20c0102610 msgr2=0x7f20c01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.304+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 shutdown_connections 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:14.304+0000 7f20c6200700 1 -- 172.21.15.46:0/1145794477 wait complete. 2024-04-17T07:46:15.022 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-04-17T07:46:15.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:15 smithi046 ceph-mon[31677]: from='client.? 172.21.15.46:0/966677831' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:46:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.582+0000 7f4a28c00700 1 Processor -- start 2024-04-17T07:46:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.583+0000 7f4a28c00700 1 -- start start 2024-04-17T07:46:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.583+0000 7f4a28c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a2407ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.583+0000 7f4a28c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2407f200 con 0x7f4a2407e810 2024-04-17T07:46:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.584+0000 7f4a21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a2407ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.584+0000 7f4a21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a2407ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39448/0 (socket says 172.21.15.46:39448) 2024-04-17T07:46:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.584+0000 7f4a21e00700 1 -- 172.21.15.46:0/3090341564 learned_addr learned my addr 172.21.15.46:0/3090341564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.584+0000 7f4a21e00700 1 -- 172.21.15.46:0/3090341564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2407fa60 con 0x7f4a2407e810 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.584+0000 7f4a21e00700 1 --2- 172.21.15.46:0/3090341564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a2407ec30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4a140098a0 tx=0x7f4a14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8557439833ca1d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.585+0000 7f4a21400700 1 -- 172.21.15.46:0/3090341564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a14004160 con 0x7f4a2407e810 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.585+0000 7f4a21400700 1 -- 172.21.15.46:0/3090341564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a14004300 con 0x7f4a2407e810 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.585+0000 7f4a21400700 1 -- 172.21.15.46:0/3090341564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a14011b50 con 0x7f4a2407e810 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.585+0000 7f4a28c00700 1 -- 172.21.15.46:0/3090341564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 msgr2=0x7f4a2407ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.585+0000 7f4a28c00700 1 --2- 172.21.15.46:0/3090341564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a2407ec30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4a140098a0 tx=0x7f4a14009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.586+0000 7f4a28c00700 1 -- 172.21.15.46:0/3090341564 shutdown_connections 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.586+0000 7f4a28c00700 1 --2- 172.21.15.46:0/3090341564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a2407ec30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.586+0000 7f4a28c00700 1 -- 172.21.15.46:0/3090341564 >> 172.21.15.46:0/3090341564 conn(0x7f4a2407d9c0 msgr2=0x7f4a24108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.586+0000 7f4a28c00700 1 -- 172.21.15.46:0/3090341564 shutdown_connections 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.586+0000 7f4a28c00700 1 -- 172.21.15.46:0/3090341564 wait complete. 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.587+0000 7f4a28c00700 1 Processor -- start 2024-04-17T07:46:16.383 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.587+0000 7f4a28c00700 1 -- start start 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.587+0000 7f4a28c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a241a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.587+0000 7f4a28c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24080e20 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.588+0000 7f4a21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a241a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.588+0000 7f4a21e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a241a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39454/0 (socket says 172.21.15.46:39454) 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.588+0000 7f4a21e00700 1 -- 172.21.15.46:0/966677831 learned_addr learned my addr 172.21.15.46:0/966677831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.588+0000 7f4a21e00700 1 -- 172.21.15.46:0/966677831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a241a2e10 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.588+0000 7f4a21e00700 1 --2- 172.21.15.46:0/966677831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a241a28d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a14011b60 tx=0x7f4a14011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.589+0000 7f4a1b400700 1 -- 172.21.15.46:0/966677831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a14003970 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.589+0000 7f4a1b400700 1 -- 172.21.15.46:0/966677831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a14003b10 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.589+0000 7f4a1b400700 1 -- 172.21.15.46:0/966677831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a14024610 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.589+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a241a30a0 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.589+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a241a5d90 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.589+0000 7f4a1b400700 1 -- 172.21.15.46:0/966677831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a14024c50 con 0x7f4a2407e810 2024-04-17T07:46:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.589+0000 7f4a1b400700 1 -- 172.21.15.46:0/966677831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4a14011e00 con 0x7f4a2407e810 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.590+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a240640b0 con 0x7f4a2407e810 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.594+0000 7f4a1b400700 1 -- 172.21.15.46:0/966677831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f4a1401a9f0 con 0x7f4a2407e810 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.672+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4a2402e250 con 0x7f4a2407e810 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.673+0000 7f4a1b400700 1 -- 172.21.15.46:0/966677831 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f4a1401a430 con 0x7f4a2407e810 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.675+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 msgr2=0x7f4a241a28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.675+0000 7f4a28c00700 1 --2- 172.21.15.46:0/966677831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a241a28d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a14011b60 tx=0x7f4a14011c40 comp rx=0 tx=0).stop 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.675+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 shutdown_connections 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.675+0000 7f4a28c00700 1 --2- 172.21.15.46:0/966677831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2407e810 0x7f4a241a28d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.675+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 >> 172.21.15.46:0/966677831 conn(0x7f4a2407d9c0 msgr2=0x7f4a24199540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.675+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 shutdown_connections 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:15.675+0000 7f4a28c00700 1 -- 172.21.15.46:0/966677831 wait complete. 2024-04-17T07:46:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-04-17T07:46:16.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:16 smithi046 systemd[1]: Stopping Ceph mon.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:46:17.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a[31673]: 2024-04-17T07:46:16.987+0000 7fa287e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:46:17.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a[31673]: 2024-04-17T07:46:16.987+0000 7fa287e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-17T07:46:17.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:17 smithi046 podman[31918]: 2024-04-17 07:46:17.122376063 +0000 UTC m=+0.323506162 container died 241e8f68d1c535b884d025c50ad9d76ab812d9f617188e554e714df028726fa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-17T07:46:18.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:17 smithi046 podman[31918]: 2024-04-17 07:46:17.896922946 +0000 UTC m=+1.098053075 container cleanup 241e8f68d1c535b884d025c50ad9d76ab812d9f617188e554e714df028726fa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-17T07:46:18.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:17 smithi046 bash[31918]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a 2024-04-17T07:46:18.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:18 smithi046 podman[31932]: 2024-04-17 07:46:18.097638699 +0000 UTC m=+0.961026862 container remove 241e8f68d1c535b884d025c50ad9d76ab812d9f617188e554e714df028726fa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-17T07:46:18.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:18 smithi046 systemd[1]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-17T07:46:18.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:18 smithi046 systemd[1]: Stopped Ceph mon.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:46:18.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:18 smithi046 systemd[1]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a.service: Consumed 2.035s CPU time. 2024-04-17T07:46:18.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:18 smithi046 systemd[1]: Starting Ceph mon.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:46:19.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 podman[32019]: 2024-04-17 07:46:18.912443738 +0000 UTC m=+0.036478144 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:46:19.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 podman[32019]: 2024-04-17 07:46:19.048326868 +0000 UTC m=+0.172361336 container create 18918c5ec8f3d10f77477ecf6e042ce5fe4f06c1f6cfa3c6e0e563988918d675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-17T07:46:19.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 podman[32019]: 2024-04-17 07:46:19.251740531 +0000 UTC m=+0.375774981 container init 18918c5ec8f3d10f77477ecf6e042ce5fe4f06c1f6cfa3c6e0e563988918d675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-17T07:46:19.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 podman[32019]: 2024-04-17 07:46:19.264485078 +0000 UTC m=+0.388519507 container start 18918c5ec8f3d10f77477ecf6e042ce5fe4f06c1f6cfa3c6e0e563988918d675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-17T07:46:19.382 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: set uid:gid to 167:167 (ceph:ceph) 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: pidfile_write: ignore empty --pid-file 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: load: jerasure load: lrc 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: RocksDB version: 7.9.2 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Git sha 0 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: DB SUMMARY 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: DB Session ID: HKYBP5PHUDQSMVZVYOOT 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: CURRENT file: CURRENT 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: IDENTITY file: IDENTITY 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-17T07:46:19.533 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104074 ; 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.error_if_exists: 0 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.create_if_missing: 0 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.paranoid_checks: 1 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.env: 0x5557fb834cc0 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.fs: PosixFileSystem 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.info_log: 0x55583b62bc60 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_file_opening_threads: 16 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.statistics: (nil) 2024-04-17T07:46:19.534 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.use_fsync: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_log_file_size: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.keep_log_file_num: 1000 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.recycle_log_file_num: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.allow_fallocate: 1 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.allow_mmap_reads: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.allow_mmap_writes: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.use_direct_reads: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.create_missing_column_families: 0 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.db_log_dir: 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.wal_dir: 2024-04-17T07:46:19.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-17T07:46:19.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-17T07:46:19.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-17T07:46:19.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-17T07:46:19.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.advise_random_on_open: 1 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.db_write_buffer_size: 0 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.write_buffer_manager: 0x55583b5fa1e0 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.rate_limiter: (nil) 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.wal_recovery_mode: 2 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.enable_thread_tracking: 0 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.enable_pipelined_write: 0 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.unordered_write: 0 2024-04-17T07:46:19.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.row_cache: None 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.wal_filter: None 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.allow_ingest_behind: 0 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.two_write_queues: 0 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.manual_wal_flush: 0 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.wal_compression: 0 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.atomic_flush: 0 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-17T07:46:19.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.log_readahead_size: 0 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.best_efforts_recovery: 0 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.allow_data_in_errors: 0 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.db_host_id: __hostname__ 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_background_jobs: 2 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_background_compactions: -1 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_subcompactions: 1 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-17T07:46:19.539 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_total_wal_size: 0 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_open_files: -1 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bytes_per_sync: 0 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_readahead_size: 0 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_background_flushes: -1 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Compression algorithms supported: 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kZSTD supported: 0 2024-04-17T07:46:19.540 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kXpressCompression supported: 0 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kLZ4HCCompression supported: 1 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kZlibCompression supported: 1 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kSnappyCompression supported: 1 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kLZ4Compression supported: 1 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: kBZip2Compression supported: 0 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.merge_operator: 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_filter: None 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_filter_factory: None 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.sst_partitioner_factory: None 2024-04-17T07:46:19.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55583b62be20) 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 4 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-04-17T07:46:19.542 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x55583b62d090 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-04-17T07:46:19.543 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: bloomfilter 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 5 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: max_auto_readahead_size: 262144 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: prepopulate_block_cache: 0 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: initial_auto_readahead_size: 8192 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.write_buffer_size: 33554432 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_write_buffer_number: 2 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression: NoCompression 2024-04-17T07:46:19.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression: Disabled 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.prefix_extractor: nullptr 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.num_levels: 7 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-17T07:46:19.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:46:19.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-17T07:46:19.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-17T07:46:19.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.level: 32767 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.strategy: 0 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.enabled: false 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-17T07:46:19.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.target_file_size_base: 67108864 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-17T07:46:19.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.arena_block_size: 1048576 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.disable_auto_compactions: 0 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-17T07:46:19.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.inplace_update_support: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.bloom_locality: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.max_successive_merges: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.paranoid_file_checks: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.force_consistency_checks: 1 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.report_bg_io_stats: 0 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.ttl: 2592000 2024-04-17T07:46:19.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.enable_blob_files: false 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.min_blob_size: 0 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.blob_file_size: 268435456 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.blob_file_starting_level: 0 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-17T07:46:19.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 642d5813-d9a6-48d6-8677-a81a796bfcb9 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713339979323189, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713339979325160, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97253, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13923, "raw_average_key_size": 49, "raw_value_size": 89636, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713339979, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "642d5813-d9a6-48d6-8677-a81a796bfcb9", "db_session_id": "HKYBP5PHUDQSMVZVYOOT", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713339979325352, "job": 1, "event": "recovery_finished"} 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55583b71a000 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: DB pointer 0x55583b704000 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:46:19.552 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 98.83 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 98.83 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-04-17T07:46:19.553 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 73.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-17T07:46:19.554 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 13.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 13.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache BinnedLRUCache@0x55583b62d090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???) e1 preinit fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???).mds e1 new map 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???).mds e1 print_map 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:46:19.555 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 bash[32019]: 18918c5ec8f3d10f77477ecf6e042ce5fe4f06c1f6cfa3c6e0e563988918d675 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 systemd[1]: Started Ceph mon.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: monmap epoch 1 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: last_changed 2024-04-17T07:46:07.693966+0000 2024-04-17T07:46:19.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:46:19.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: min_mon_release 19 (squid) 2024-04-17T07:46:19.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: election_strategy: 1 2024-04-17T07:46:19.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:46:19.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: fsmap 2024-04-17T07:46:19.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: osdmap e1: 0 total, 0 up, 0 in 2024-04-17T07:46:19.557 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:19 smithi046 ceph-mon[32038]: mgrmap e1: no daemons active 2024-04-17T07:46:20.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.070+0000 7f83cfe00700 1 Processor -- start 2024-04-17T07:46:20.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.070+0000 7f83cfe00700 1 -- start start 2024-04-17T07:46:20.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.071+0000 7f83cfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d010a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.071+0000 7f83cfe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83d010ad20 con 0x7f83d010a330 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.072+0000 7f83cea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d010a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.072+0000 7f83cea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d010a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33764/0 (socket says 172.21.15.46:33764) 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.072+0000 7f83cea00700 1 -- 172.21.15.46:0/1874323782 learned_addr learned my addr 172.21.15.46:0/1874323782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.072+0000 7f83cea00700 1 -- 172.21.15.46:0/1874323782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d010b530 con 0x7f83d010a330 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.073+0000 7f83cea00700 1 --2- 172.21.15.46:0/1874323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d010a750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83c00098a0 tx=0x7f83c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d710d66748985d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.073+0000 7f83cd600700 1 -- 172.21.15.46:0/1874323782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0004160 con 0x7f83d010a330 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.073+0000 7f83cd600700 1 -- 172.21.15.46:0/1874323782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f83c0004300 con 0x7f83d010a330 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.073+0000 7f83cd600700 1 -- 172.21.15.46:0/1874323782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0011b50 con 0x7f83d010a330 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.074+0000 7f83cfe00700 1 -- 172.21.15.46:0/1874323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 msgr2=0x7f83d010a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.074+0000 7f83cfe00700 1 --2- 172.21.15.46:0/1874323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d010a750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83c00098a0 tx=0x7f83c0009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.074+0000 7f83cfe00700 1 -- 172.21.15.46:0/1874323782 shutdown_connections 2024-04-17T07:46:20.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.074+0000 7f83cfe00700 1 --2- 172.21.15.46:0/1874323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d010a750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.074+0000 7f83cfe00700 1 -- 172.21.15.46:0/1874323782 >> 172.21.15.46:0/1874323782 conn(0x7f83d007d9c0 msgr2=0x7f83d0108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.074+0000 7f83cfe00700 1 -- 172.21.15.46:0/1874323782 shutdown_connections 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.075+0000 7f83cfe00700 1 -- 172.21.15.46:0/1874323782 wait complete. 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.076+0000 7f83cfe00700 1 Processor -- start 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.076+0000 7f83cfe00700 1 -- start start 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.076+0000 7f83cfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.076+0000 7f83cfe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83d010e8e0 con 0x7f83d010a330 2024-04-17T07:46:20.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.077+0000 7f83cea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.077+0000 7f83cea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33780/0 (socket says 172.21.15.46:33780) 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.077+0000 7f83cea00700 1 -- 172.21.15.46:0/1374110371 learned_addr learned my addr 172.21.15.46:0/1374110371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.077+0000 7f83cea00700 1 -- 172.21.15.46:0/1374110371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d019e840 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.077+0000 7f83cea00700 1 --2- 172.21.15.46:0/1374110371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d019e300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f83c0011670 tx=0x7f83c0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.078+0000 7f83c7400700 1 -- 172.21.15.46:0/1374110371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c00039c0 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.078+0000 7f83c7400700 1 -- 172.21.15.46:0/1374110371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f83c0011e50 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.078+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d019ead0 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.078+0000 7f83c7400700 1 -- 172.21.15.46:0/1374110371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c001a5c0 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.078+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d01a17c0 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.078+0000 7f83c7400700 1 -- 172.21.15.46:0/1374110371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f83c001ac50 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.078+0000 7f83c7400700 1 -- 172.21.15.46:0/1374110371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f83c0019700 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.079+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d00640b0 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.082+0000 7f83c7400700 1 -- 172.21.15.46:0/1374110371 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f83c0003b60 con 0x7f83d010a330 2024-04-17T07:46:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.160+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f83d01a1990 con 0x7f83d010a330 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.164+0000 7f83c7400700 1 -- 172.21.15.46:0/1374110371 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f83c0019900 con 0x7f83d010a330 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.165+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 msgr2=0x7f83d019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.165+0000 7f83cfe00700 1 --2- 172.21.15.46:0/1374110371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d019e300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f83c0011670 tx=0x7f83c0011b60 comp rx=0 tx=0).stop 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.165+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 shutdown_connections 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.165+0000 7f83cfe00700 1 --2- 172.21.15.46:0/1374110371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83d010a330 0x7f83d019e300 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.165+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 >> 172.21.15.46:0/1374110371 conn(0x7f83d007d9c0 msgr2=0x7f83d0107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.166+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 shutdown_connections 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:20.166+0000 7f83cfe00700 1 -- 172.21.15.46:0/1374110371 wait complete. 2024-04-17T07:46:20.748 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-17T07:46:20.764 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:46:20.764 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-04-17T07:46:20.766 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-17T07:46:20.767 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-17T07:46:21.184 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.x 2024-04-17T07:46:21.184 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.x.service: Unit ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.x.service not loaded. 2024-04-17T07:46:21.455 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968.target.wants/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@.service. 2024-04-17T07:46:21.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:21 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1374110371' entity='client.admin' 2024-04-17T07:46:21.488 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:21 smithi046 systemd[1]: Starting Ceph mgr.x for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:46:22.105 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:21 smithi046 podman[32251]: 2024-04-17 07:46:21.850866127 +0000 UTC m=+0.126612804 container create 4e3930768b4a74e4144d593eb544ac006578513fae0a5086a3a75968442ede14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-17T07:46:22.106 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:21 smithi046 podman[32251]: 2024-04-17 07:46:21.758053165 +0000 UTC m=+0.033799805 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:46:22.106 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:22 smithi046 podman[32251]: 2024-04-17 07:46:22.095627261 +0000 UTC m=+0.371373945 container init 4e3930768b4a74e4144d593eb544ac006578513fae0a5086a3a75968442ede14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-17T07:46:22.106 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:22 smithi046 podman[32251]: 2024-04-17 07:46:22.105573658 +0000 UTC m=+0.381320341 container start 4e3930768b4a74e4144d593eb544ac006578513fae0a5086a3a75968442ede14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-17T07:46:22.216 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:46:22.216 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-04-17T07:46:22.233 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:46:22.233 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-04-17T07:46:22.233 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-04-17T07:46:22.233 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-04-17T07:46:22.254 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:46:22.254 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-04-17T07:46:22.272 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:46:22.272 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-04-17T07:46:22.272 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-04-17T07:46:22.272 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-17T07:46:22.272 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-04-17T07:46:22.272 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-04-17T07:46:22.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:22 smithi046 bash[32251]: 4e3930768b4a74e4144d593eb544ac006578513fae0a5086a3a75968442ede14 2024-04-17T07:46:22.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:22 smithi046 systemd[1]: Started Ceph mgr.x for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:46:22.964 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:22 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:22.914+0000 7fdd39f1e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:46:23.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:23 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4142295746' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:46:23.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:23 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:23.056+0000 7fdd39f1e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:46:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:46:23.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:46:23.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:46:23.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:46:23.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:46:23.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:46:23.735 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:46:23.737 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:46:11.977289+0000", 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.870+0000 7f260a400700 1 Processor -- start 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.870+0000 7f260a400700 1 -- start start 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.871+0000 7f260a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.871+0000 7f260a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f260407c8c0 con 0x7f260407de40 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.871+0000 7f2603400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.872+0000 7f2603400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33804/0 (socket says 172.21.15.46:33804) 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.872+0000 7f2603400700 1 -- 172.21.15.46:0/1521451077 learned_addr learned my addr 172.21.15.46:0/1521451077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:23.738 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.872+0000 7f2603400700 1 -- 172.21.15.46:0/1521451077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260407ca40 con 0x7f260407de40 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.872+0000 7f2603400700 1 --2- 172.21.15.46:0/1521451077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260407c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f25f40098a0 tx=0x7f25f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2a96a47b66679a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.873+0000 7f2602000700 1 -- 172.21.15.46:0/1521451077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f4004160 con 0x7f260407de40 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.873+0000 7f2602000700 1 -- 172.21.15.46:0/1521451077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25f4004300 con 0x7f260407de40 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.873+0000 7f2602000700 1 -- 172.21.15.46:0/1521451077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f4011b50 con 0x7f260407de40 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.874+0000 7f260a400700 1 -- 172.21.15.46:0/1521451077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 msgr2=0x7f260407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.874+0000 7f260a400700 1 --2- 172.21.15.46:0/1521451077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260407c2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f25f40098a0 tx=0x7f25f4009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.874+0000 7f260a400700 1 -- 172.21.15.46:0/1521451077 shutdown_connections 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.874+0000 7f260a400700 1 --2- 172.21.15.46:0/1521451077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260407c2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.874+0000 7f260a400700 1 -- 172.21.15.46:0/1521451077 >> 172.21.15.46:0/1521451077 conn(0x7f26041039b0 msgr2=0x7f2604105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.874+0000 7f260a400700 1 -- 172.21.15.46:0/1521451077 shutdown_connections 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.874+0000 7f260a400700 1 -- 172.21.15.46:0/1521451077 wait complete. 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.875+0000 7f260a400700 1 Processor -- start 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.876+0000 7f260a400700 1 -- start start 2024-04-17T07:46:23.739 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.876+0000 7f260a400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.876+0000 7f260a400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f260410e960 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.876+0000 7f2603400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.876+0000 7f2603400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33820/0 (socket says 172.21.15.46:33820) 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.876+0000 7f2603400700 1 -- 172.21.15.46:0/4142295746 learned_addr learned my addr 172.21.15.46:0/4142295746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.877+0000 7f2603400700 1 -- 172.21.15.46:0/4142295746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260419ea40 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.877+0000 7f2603400700 1 --2- 172.21.15.46:0/4142295746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260419e500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f25f4011b60 tx=0x7f25f4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.877+0000 7f25fbe00700 1 -- 172.21.15.46:0/4142295746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f4003970 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.877+0000 7f25fbe00700 1 -- 172.21.15.46:0/4142295746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25f4003b10 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.877+0000 7f25fbe00700 1 -- 172.21.15.46:0/4142295746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f4024610 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.878+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260419ecd0 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.878+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a19c0 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.878+0000 7f25fbe00700 1 -- 172.21.15.46:0/4142295746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f25f4024b40 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.878+0000 7f25fbe00700 1 -- 172.21.15.46:0/4142295746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25f4011e00 con 0x7f260407de40 2024-04-17T07:46:23.740 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.879+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26040640b0 con 0x7f260407de40 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.883+0000 7f25fbe00700 1 -- 172.21.15.46:0/4142295746 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f25f401a9f0 con 0x7f260407de40 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.963+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f26041a1f80 con 0x7f260407de40 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.964+0000 7f25fbe00700 1 -- 172.21.15.46:0/4142295746 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f25f401a7c0 con 0x7f260407de40 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.966+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 msgr2=0x7f260419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.966+0000 7f260a400700 1 --2- 172.21.15.46:0/4142295746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260419e500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f25f4011b60 tx=0x7f25f4011c40 comp rx=0 tx=0).stop 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.966+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 shutdown_connections 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.966+0000 7f260a400700 1 --2- 172.21.15.46:0/4142295746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f260407de40 0x7f260419e500 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.966+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 >> 172.21.15.46:0/4142295746 conn(0x7f26041039b0 msgr2=0x7f2604104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.966+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 shutdown_connections 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:22.966+0000 7f260a400700 1 -- 172.21.15.46:0/4142295746 wait complete. 2024-04-17T07:46:23.741 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-04-17T07:46:24.245 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:23 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:23.932+0000 7fdd39f1e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:46:24.245 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:24.059+0000 7fdd39f1e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:46:24.245 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:24.244+0000 7fdd39f1e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:46:24.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:24.375+0000 7fdd39f1e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:46:24.611 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:24.488+0000 7fdd39f1e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:46:25.096 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:24.834+0000 7fdd39f1e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:46:25.097 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:24.980+0000 7fdd39f1e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:46:25.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:25.096+0000 7fdd39f1e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:46:25.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:25.224+0000 7fdd39f1e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:46:25.860 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:25.447+0000 7fdd39f1e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:46:26.732 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:26 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2330932475' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:46:27.015 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:26 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:26.833+0000 7fdd39f1e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:46:27.015 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:27.015+0000 7fdd39f1e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:46:27.031 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:27.044 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:27.045 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:46:27.046 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:46:27.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:46:27.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:46:11.977289+0000", 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.296+0000 7ff4acc00700 1 Processor -- start 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.296+0000 7ff4acc00700 1 -- start start 2024-04-17T07:46:27.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.296+0000 7ff4acc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a810b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.296+0000 7ff4acc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a807be00 con 0x7ff4a8108e00 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.297+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a810b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.297+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a810b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33830/0 (socket says 172.21.15.46:33830) 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.297+0000 7ff4a5e00700 1 -- 172.21.15.46:0/3818496541 learned_addr learned my addr 172.21.15.46:0/3818496541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.297+0000 7ff4a5e00700 1 -- 172.21.15.46:0/3818496541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a810b730 con 0x7ff4a8108e00 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.298+0000 7ff4a5e00700 1 --2- 172.21.15.46:0/3818496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a810b1f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0676a3d297fcac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.298+0000 7ff4a5400700 1 -- 172.21.15.46:0/3818496541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498004160 con 0x7ff4a8108e00 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.298+0000 7ff4a5400700 1 -- 172.21.15.46:0/3818496541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff498004300 con 0x7ff4a8108e00 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.298+0000 7ff4a5400700 1 -- 172.21.15.46:0/3818496541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498011b50 con 0x7ff4a8108e00 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.299+0000 7ff4acc00700 1 -- 172.21.15.46:0/3818496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 msgr2=0x7ff4a810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.299+0000 7ff4acc00700 1 --2- 172.21.15.46:0/3818496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a810b1f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.299+0000 7ff4acc00700 1 -- 172.21.15.46:0/3818496541 shutdown_connections 2024-04-17T07:46:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.299+0000 7ff4acc00700 1 --2- 172.21.15.46:0/3818496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a810b1f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.299+0000 7ff4acc00700 1 -- 172.21.15.46:0/3818496541 >> 172.21.15.46:0/3818496541 conn(0x7ff4a8102830 msgr2=0x7ff4a8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.299+0000 7ff4acc00700 1 -- 172.21.15.46:0/3818496541 shutdown_connections 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.299+0000 7ff4acc00700 1 -- 172.21.15.46:0/3818496541 wait complete. 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.300+0000 7ff4acc00700 1 Processor -- start 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.301+0000 7ff4acc00700 1 -- start start 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.301+0000 7ff4acc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.301+0000 7ff4acc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810c700 con 0x7ff4a8108e00 2024-04-17T07:46:27.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.301+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:27.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.301+0000 7ff4a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a819e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33838/0 (socket says 172.21.15.46:33838) 2024-04-17T07:46:27.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.301+0000 7ff4a5e00700 1 -- 172.21.15.46:0/2330932475 learned_addr learned my addr 172.21.15.46:0/2330932475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:27.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.302+0000 7ff4a5e00700 1 -- 172.21.15.46:0/2330932475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a819e9e0 con 0x7ff4a8108e00 2024-04-17T07:46:27.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.302+0000 7ff4a5e00700 1 --2- 172.21.15.46:0/2330932475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a819e4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff498011b60 tx=0x7ff498011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:27.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.302+0000 7ff49f400700 1 -- 172.21.15.46:0/2330932475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498003b20 con 0x7ff4a8108e00 2024-04-17T07:46:27.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.302+0000 7ff49f400700 1 -- 172.21.15.46:0/2330932475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff498003cc0 con 0x7ff4a8108e00 2024-04-17T07:46:27.053 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.302+0000 7ff49f400700 1 -- 172.21.15.46:0/2330932475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498024840 con 0x7ff4a8108e00 2024-04-17T07:46:27.053 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.302+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a819ec70 con 0x7ff4a8108e00 2024-04-17T07:46:27.053 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.302+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81a1960 con 0x7ff4a8108e00 2024-04-17T07:46:27.053 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.303+0000 7ff49f400700 1 -- 172.21.15.46:0/2330932475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff498022020 con 0x7ff4a8108e00 2024-04-17T07:46:27.053 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.303+0000 7ff49f400700 1 -- 172.21.15.46:0/2330932475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff498011e00 con 0x7ff4a8108e00 2024-04-17T07:46:27.053 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.304+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a80640b0 con 0x7ff4a8108e00 2024-04-17T07:46:27.054 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.306+0000 7ff49f400700 1 -- 172.21.15.46:0/2330932475 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7ff498043e50 con 0x7ff4a8108e00 2024-04-17T07:46:27.054 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.387+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff4a81a1ea0 con 0x7ff4a8108e00 2024-04-17T07:46:27.054 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.387+0000 7ff49f400700 1 -- 172.21.15.46:0/2330932475 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff498032030 con 0x7ff4a8108e00 2024-04-17T07:46:27.054 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.389+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 msgr2=0x7ff4a819e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:27.054 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.389+0000 7ff4acc00700 1 --2- 172.21.15.46:0/2330932475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a819e4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff498011b60 tx=0x7ff498011c40 comp rx=0 tx=0).stop 2024-04-17T07:46:27.055 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.389+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 shutdown_connections 2024-04-17T07:46:27.055 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.389+0000 7ff4acc00700 1 --2- 172.21.15.46:0/2330932475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4a8108e00 0x7ff4a819e4a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:27.055 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.389+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 >> 172.21.15.46:0/2330932475 conn(0x7ff4a8102830 msgr2=0x7ff4a81032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:27.055 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.390+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 shutdown_connections 2024-04-17T07:46:27.055 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:26.390+0000 7ff4acc00700 1 -- 172.21.15.46:0/2330932475 wait complete. 2024-04-17T07:46:27.055 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-04-17T07:46:27.840 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:27.447+0000 7fdd39f1e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:46:27.840 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:27.570+0000 7fdd39f1e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:46:28.096 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:27.839+0000 7fdd39f1e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:46:28.096 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:28.094+0000 7fdd39f1e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:46:28.808 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:28.485+0000 7fdd39f1e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:46:28.809 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:28.807+0000 7fdd39f1e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:46:29.419 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:29 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:29.136+0000 7fdd39f1e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:46:30.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:29 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/366751806' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:46:30.437 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:30.246+0000 7fdd39f1e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:46:30.453 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:30.453 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:46:30.453 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:46:30.454 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:46:30.454 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:46:30.454 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:46:30.454 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:46:30.455 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:30.455 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:46:30.455 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:46:30.455 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:46:30.456 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:30.457 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:46:30.457 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:46:30.457 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:30.457 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-17T07:46:30.457 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:46:30.457 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:30.458 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:46:30.458 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:46:30.458 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:30.458 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:46:30.458 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:30.458 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:46:30.460 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:46:30.460 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:46:30.460 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:46:30.460 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:46:30.460 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:46:30.460 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:30.460 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:46:30.461 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:46:30.461 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:46:30.461 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:46:30.461 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:46:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:30.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:30.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:30.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:46:30.465 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:46:11.977289+0000", 2024-04-17T07:46:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:46:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:46:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.664+0000 7f6ab0e00700 1 Processor -- start 2024-04-17T07:46:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.664+0000 7f6ab0e00700 1 -- start start 2024-04-17T07:46:30.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.665+0000 7f6ab0e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac107030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:30.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.665+0000 7f6ab0e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac07bde0 con 0x7f6aac106c10 2024-04-17T07:46:30.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.665+0000 7f6aa9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:30.468 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.665+0000 7f6aa9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42788/0 (socket says 172.21.15.46:42788) 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.665+0000 7f6aa9e00700 1 -- 172.21.15.46:0/2743742252 learned_addr learned my addr 172.21.15.46:0/2743742252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.666+0000 7f6aa9e00700 1 -- 172.21.15.46:0/2743742252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac107570 con 0x7f6aac106c10 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.666+0000 7f6aa9e00700 1 --2- 172.21.15.46:0/2743742252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac107030 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0098a0 tx=0x7f6a9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da388430fffeb97e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.666+0000 7f6aa8a00700 1 -- 172.21.15.46:0/2743742252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a9c004160 con 0x7f6aac106c10 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.666+0000 7f6aa8a00700 1 -- 172.21.15.46:0/2743742252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a9c004300 con 0x7f6aac106c10 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.666+0000 7f6aa8a00700 1 -- 172.21.15.46:0/2743742252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a9c011b50 con 0x7f6aac106c10 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.667+0000 7f6ab0e00700 1 -- 172.21.15.46:0/2743742252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 msgr2=0x7f6aac107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:30.469 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.667+0000 7f6ab0e00700 1 --2- 172.21.15.46:0/2743742252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac107030 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0098a0 tx=0x7f6a9c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:30.470 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.667+0000 7f6ab0e00700 1 -- 172.21.15.46:0/2743742252 shutdown_connections 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.667+0000 7f6ab0e00700 1 --2- 172.21.15.46:0/2743742252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac107030 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.667+0000 7f6ab0e00700 1 -- 172.21.15.46:0/2743742252 >> 172.21.15.46:0/2743742252 conn(0x7f6aac102830 msgr2=0x7f6aac104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.667+0000 7f6ab0e00700 1 -- 172.21.15.46:0/2743742252 shutdown_connections 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.667+0000 7f6ab0e00700 1 -- 172.21.15.46:0/2743742252 wait complete. 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.668+0000 7f6ab0e00700 1 Processor -- start 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.668+0000 7f6ab0e00700 1 -- start start 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.669+0000 7f6ab0e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:30.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.669+0000 7f6ab0e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac108600 con 0x7f6aac106c10 2024-04-17T07:46:30.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.669+0000 7f6aa9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:30.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.669+0000 7f6aa9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42800/0 (socket says 172.21.15.46:42800) 2024-04-17T07:46:30.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.669+0000 7f6aa9e00700 1 -- 172.21.15.46:0/366751806 learned_addr learned my addr 172.21.15.46:0/366751806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:30.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.670+0000 7f6aa9e00700 1 -- 172.21.15.46:0/366751806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac19e9c0 con 0x7f6aac106c10 2024-04-17T07:46:30.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.670+0000 7f6aa9e00700 1 --2- 172.21.15.46:0/366751806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac19e480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c011b60 tx=0x7f6a9c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:30.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.670+0000 7f6aa2a00700 1 -- 172.21.15.46:0/366751806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a9c003b20 con 0x7f6aac106c10 2024-04-17T07:46:30.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.670+0000 7f6aa2a00700 1 -- 172.21.15.46:0/366751806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a9c003cc0 con 0x7f6aac106c10 2024-04-17T07:46:30.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.670+0000 7f6aa2a00700 1 -- 172.21.15.46:0/366751806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a9c024840 con 0x7f6aac106c10 2024-04-17T07:46:30.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.670+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac19ec50 con 0x7f6aac106c10 2024-04-17T07:46:30.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.670+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac1a1940 con 0x7f6aac106c10 2024-04-17T07:46:30.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.671+0000 7f6aa2a00700 1 -- 172.21.15.46:0/366751806 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6a9c022020 con 0x7f6aac106c10 2024-04-17T07:46:30.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.671+0000 7f6aa2a00700 1 -- 172.21.15.46:0/366751806 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6a9c011e00 con 0x7f6aac106c10 2024-04-17T07:46:30.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.672+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac0640b0 con 0x7f6aac106c10 2024-04-17T07:46:30.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.674+0000 7f6aa2a00700 1 -- 172.21.15.46:0/366751806 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f6a9c043e50 con 0x7f6aac106c10 2024-04-17T07:46:30.475 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.754+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6aac0008d0 con 0x7f6aac106c10 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.755+0000 7f6aa2a00700 1 -- 172.21.15.46:0/366751806 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f6a9c032020 con 0x7f6aac106c10 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.756+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 msgr2=0x7f6aac19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.756+0000 7f6ab0e00700 1 --2- 172.21.15.46:0/366751806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac19e480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c011b60 tx=0x7f6a9c011c40 comp rx=0 tx=0).stop 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.756+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 shutdown_connections 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.756+0000 7f6ab0e00700 1 --2- 172.21.15.46:0/366751806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aac106c10 0x7f6aac19e480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.756+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 >> 172.21.15.46:0/366751806 conn(0x7f6aac102830 msgr2=0x7f6aac195120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.756+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 shutdown_connections 2024-04-17T07:46:30.476 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:29.756+0000 7f6ab0e00700 1 -- 172.21.15.46:0/366751806 wait complete. 2024-04-17T07:46:30.477 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (3/15)... 2024-04-17T07:46:31.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:30.970+0000 7fdd39f1e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:46:31.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:31.105+0000 7fdd39f1e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:46:31.860 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:31.400+0000 7fdd39f1e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:46:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: Activating manager daemon x 2024-04-17T07:46:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: mgrmap e2: x(active, starting, since 0.0119739s) 2024-04-17T07:46:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T07:46:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T07:46:31.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T07:46:31.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:46:31.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T07:46:31.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: Manager daemon x is now available 2024-04-17T07:46:31.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T07:46:31.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:31 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T07:46:32.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:32 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' 2024-04-17T07:46:32.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:32 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' 2024-04-17T07:46:32.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:32 smithi046 ceph-mon[32038]: from='mgr.14100 172.21.15.46:0/3182434001' entity='mgr.x' 2024-04-17T07:46:32.789 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:32 smithi046 ceph-mon[32038]: mgrmap e3: x(active, since 1.02126s) 2024-04-17T07:46:33.728 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:33 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1758056371' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:46:34.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:34.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:46:34.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:46:34.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:46:34.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:46:34.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:46:34.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:46:34.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:34.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:46:34.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:46:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:46:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:46:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:46:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-17T07:46:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:46:34.068 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:34.068 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:46:34.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:46:34.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:46:34.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:46:34.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:34.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:46:34.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:46:34.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:46:34.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:46:34.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:34.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:46:34.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T07:46:34.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:46:34.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:46:34.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:46:11.977289+0000", 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.085+0000 7f0ed8e00700 1 Processor -- start 2024-04-17T07:46:34.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.085+0000 7f0ed8e00700 1 -- start start 2024-04-17T07:46:34.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.086+0000 7f0ed8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed410b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:34.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.086+0000 7f0ed8e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed407be60 con 0x7f0ed4108db0 2024-04-17T07:46:34.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.086+0000 7f0ed1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed410b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:34.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.087+0000 7f0ed1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed410b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42878/0 (socket says 172.21.15.46:42878) 2024-04-17T07:46:34.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.087+0000 7f0ed1e00700 1 -- 172.21.15.46:0/576665131 learned_addr learned my addr 172.21.15.46:0/576665131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:34.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.087+0000 7f0ed1e00700 1 -- 172.21.15.46:0/576665131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed410b6e0 con 0x7f0ed4108db0 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.088+0000 7f0ed1e00700 1 --2- 172.21.15.46:0/576665131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed410b1a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40098a0 tx=0x7f0ec4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eae9fa9ba309374 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.088+0000 7f0ed1400700 1 -- 172.21.15.46:0/576665131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ec4004160 con 0x7f0ed4108db0 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.088+0000 7f0ed1400700 1 -- 172.21.15.46:0/576665131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ec4004300 con 0x7f0ed4108db0 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.088+0000 7f0ed1400700 1 -- 172.21.15.46:0/576665131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ec4011b50 con 0x7f0ed4108db0 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.089+0000 7f0ed8e00700 1 -- 172.21.15.46:0/576665131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 msgr2=0x7f0ed410b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.089+0000 7f0ed8e00700 1 --2- 172.21.15.46:0/576665131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed410b1a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40098a0 tx=0x7f0ec4009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.089+0000 7f0ed8e00700 1 -- 172.21.15.46:0/576665131 shutdown_connections 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.089+0000 7f0ed8e00700 1 --2- 172.21.15.46:0/576665131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed410b1a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.089+0000 7f0ed8e00700 1 -- 172.21.15.46:0/576665131 >> 172.21.15.46:0/576665131 conn(0x7f0ed4102860 msgr2=0x7f0ed4104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.089+0000 7f0ed8e00700 1 -- 172.21.15.46:0/576665131 shutdown_connections 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.089+0000 7f0ed8e00700 1 -- 172.21.15.46:0/576665131 wait complete. 2024-04-17T07:46:34.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.090+0000 7f0ed8e00700 1 Processor -- start 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.090+0000 7f0ed8e00700 1 -- start start 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.090+0000 7f0ed8e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed4108630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.090+0000 7f0ed8e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed410c710 con 0x7f0ed4108db0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.091+0000 7f0ed1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed4108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.091+0000 7f0ed1e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed4108630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42888/0 (socket says 172.21.15.46:42888) 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.091+0000 7f0ed1e00700 1 -- 172.21.15.46:0/1758056371 learned_addr learned my addr 172.21.15.46:0/1758056371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.091+0000 7f0ed1e00700 1 -- 172.21.15.46:0/1758056371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed4108b70 con 0x7f0ed4108db0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.091+0000 7f0ed1e00700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed4108630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4011670 tx=0x7f0ec4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.092+0000 7f0ecb400700 1 -- 172.21.15.46:0/1758056371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ec40039c0 con 0x7f0ed4108db0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.092+0000 7f0ecb400700 1 -- 172.21.15.46:0/1758056371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ec4011e50 con 0x7f0ed4108db0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.092+0000 7f0ecb400700 1 -- 172.21.15.46:0/1758056371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ec401a5c0 con 0x7f0ed4108db0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.092+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed4106d60 con 0x7f0ed4108db0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.092+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed41072a0 con 0x7f0ed4108db0 2024-04-17T07:46:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.093+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed40640b0 con 0x7f0ed4108db0 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.094+0000 7f0ecb400700 1 -- 172.21.15.46:0/1758056371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f0ec401a870 con 0x7f0ed4108db0 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.094+0000 7f0ecb400700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f0eb8039510 0x7f0eb803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.094+0000 7f0ecb400700 1 -- 172.21.15.46:0/1758056371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ec404daf0 con 0x7f0ed4108db0 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.094+0000 7f0ec9400700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f0eb8039510 0x7f0eb803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.095+0000 7f0ec9400700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f0eb8039510 0x7f0eb803b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006fd0 tx=0x7f0ebc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.100+0000 7f0ecb400700 1 -- 172.21.15.46:0/1758056371 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0ec4016260 con 0x7f0ed4108db0 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.422+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0ed4107770 con 0x7f0ed4108db0 2024-04-17T07:46:34.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.423+0000 7f0ecb400700 1 -- 172.21.15.46:0/1758056371 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0ec4019df0 con 0x7f0ed4108db0 2024-04-17T07:46:34.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.425+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f0eb8039510 msgr2=0x7f0eb803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:34.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.425+0000 7f0ed8e00700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f0eb8039510 0x7f0eb803b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006fd0 tx=0x7f0ebc006e40 comp rx=0 tx=0).stop 2024-04-17T07:46:34.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 msgr2=0x7f0ed4108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:34.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed4108630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4011670 tx=0x7f0ec4011b60 comp rx=0 tx=0).stop 2024-04-17T07:46:34.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 shutdown_connections 2024-04-17T07:46:34.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f0eb8039510 0x7f0eb803b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:34.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 --2- 172.21.15.46:0/1758056371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed4108db0 0x7f0ed4108630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:34.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 >> 172.21.15.46:0/1758056371 conn(0x7f0ed4102860 msgr2=0x7f0ed4103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:34.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 shutdown_connections 2024-04-17T07:46:34.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:33.426+0000 7f0ed8e00700 1 -- 172.21.15.46:0/1758056371 wait complete. 2024-04-17T07:46:34.086 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-04-17T07:46:34.731 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:34 smithi046 ceph-mon[32038]: mgrmap e4: x(active, since 2s) 2024-04-17T07:46:35.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:35 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1951007857' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-17T07:46:35.701 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-17T07:46:35.702 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.730+0000 7f143ca00700 1 Processor -- start 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.730+0000 7f143ca00700 1 -- start start 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.731+0000 7f143ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143810a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.731+0000 7f143ca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f143810ad00 con 0x7f143810a310 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.731+0000 7f1436800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143810a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.731+0000 7f1436800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143810a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42898/0 (socket says 172.21.15.46:42898) 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.731+0000 7f1436800700 1 -- 172.21.15.46:0/2244468705 learned_addr learned my addr 172.21.15.46:0/2244468705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.732+0000 7f1436800700 1 -- 172.21.15.46:0/2244468705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143810b510 con 0x7f143810a310 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.732+0000 7f1436800700 1 --2- 172.21.15.46:0/2244468705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143810a730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f1428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bb1714c6e7e421 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f1435400700 1 -- 172.21.15.46:0/2244468705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1428004160 con 0x7f143810a310 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f1435400700 1 -- 172.21.15.46:0/2244468705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1428004300 con 0x7f143810a310 2024-04-17T07:46:35.703 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f1435400700 1 -- 172.21.15.46:0/2244468705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1428011b50 con 0x7f143810a310 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f143ca00700 1 -- 172.21.15.46:0/2244468705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 msgr2=0x7f143810a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f143ca00700 1 --2- 172.21.15.46:0/2244468705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143810a730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f1428009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f143ca00700 1 -- 172.21.15.46:0/2244468705 shutdown_connections 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f143ca00700 1 --2- 172.21.15.46:0/2244468705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143810a730 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f143ca00700 1 -- 172.21.15.46:0/2244468705 >> 172.21.15.46:0/2244468705 conn(0x7f143807d9c0 msgr2=0x7f1438108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.733+0000 7f143ca00700 1 -- 172.21.15.46:0/2244468705 shutdown_connections 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.734+0000 7f143ca00700 1 -- 172.21.15.46:0/2244468705 wait complete. 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.734+0000 7f143ca00700 1 Processor -- start 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.735+0000 7f143ca00700 1 -- start start 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.735+0000 7f143ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.735+0000 7f143ca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f143810e8c0 con 0x7f143810a310 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.735+0000 7f1436800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.735+0000 7f1436800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143819e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42908/0 (socket says 172.21.15.46:42908) 2024-04-17T07:46:35.704 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.735+0000 7f1436800700 1 -- 172.21.15.46:0/1951007857 learned_addr learned my addr 172.21.15.46:0/1951007857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.736+0000 7f1436800700 1 -- 172.21.15.46:0/1951007857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143819e800 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.736+0000 7f1436800700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143819e2c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1428011670 tx=0x7f1428011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.736+0000 7f142f400700 1 -- 172.21.15.46:0/1951007857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14280039c0 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.736+0000 7f142f400700 1 -- 172.21.15.46:0/1951007857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1428011e50 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.736+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143819ea90 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.736+0000 7f142f400700 1 -- 172.21.15.46:0/1951007857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f142801a5c0 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.737+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14381a1780 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.738+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14380640b0 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.738+0000 7f142f400700 1 -- 172.21.15.46:0/1951007857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1428024430 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.739+0000 7f142f400700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f141c039560 0x7f141c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.739+0000 7f142f400700 1 -- 172.21.15.46:0/1951007857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f142804da10 con 0x7f143810a310 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.739+0000 7f1435e00700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f141c039560 0x7f141c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:35.705 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.740+0000 7f1435e00700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f141c039560 0x7f141c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1424006fd0 tx=0x7f1424006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.744+0000 7f142f400700 1 -- 172.21.15.46:0/1951007857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f142804d430 con 0x7f143810a310 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.973+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f14381a1950 con 0x7f143810a310 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.975+0000 7f142f400700 1 -- 172.21.15.46:0/1951007857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f14381a1950 con 0x7f143810a310 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f141c039560 msgr2=0x7f141c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f141c039560 0x7f141c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1424006fd0 tx=0x7f1424006e40 comp rx=0 tx=0).stop 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 msgr2=0x7f143819e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143819e2c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1428011670 tx=0x7f1428011b60 comp rx=0 tx=0).stop 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 shutdown_connections 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f141c039560 0x7f141c03ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 --2- 172.21.15.46:0/1951007857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143810a310 0x7f143819e2c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.978+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 >> 172.21.15.46:0/1951007857 conn(0x7f143807d9c0 msgr2=0x7f1438107b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.979+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 shutdown_connections 2024-04-17T07:46:35.706 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:34.979+0000 7f143ca00700 1 -- 172.21.15.46:0/1951007857 wait complete. 2024-04-17T07:46:35.707 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-04-17T07:46:37.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:36 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3395830797' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-17T07:46:37.052 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: ignoring --setuser ceph since I am not root 2024-04-17T07:46:37.052 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: ignoring --setgroup ceph since I am not root 2024-04-17T07:46:37.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.315+0000 7f67b7e00700 1 Processor -- start 2024-04-17T07:46:37.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.316+0000 7f67b7e00700 1 -- start start 2024-04-17T07:46:37.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.316+0000 7f67b7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b010a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:37.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.316+0000 7f67b7e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010af50 con 0x7f67b010a560 2024-04-17T07:46:37.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.316+0000 7f67b5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b010a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:37.371 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.317+0000 7f67b5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b010a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42922/0 (socket says 172.21.15.46:42922) 2024-04-17T07:46:37.377 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.317+0000 7f67b5400700 1 -- 172.21.15.46:0/1768058645 learned_addr learned my addr 172.21.15.46:0/1768058645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:37.377 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.317+0000 7f67b5400700 1 -- 172.21.15.46:0/1768058645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b010b710 con 0x7f67b010a560 2024-04-17T07:46:37.377 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.317+0000 7f67b5400700 1 --2- 172.21.15.46:0/1768058645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b010a980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67a40098a0 tx=0x7f67a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4ceb82a8c0e119 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:37.377 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.318+0000 7f67afe00700 1 -- 172.21.15.46:0/1768058645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a4004160 con 0x7f67b010a560 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.318+0000 7f67afe00700 1 -- 172.21.15.46:0/1768058645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67a4004300 con 0x7f67b010a560 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.318+0000 7f67afe00700 1 -- 172.21.15.46:0/1768058645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a4011b50 con 0x7f67b010a560 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.319+0000 7f67b7e00700 1 -- 172.21.15.46:0/1768058645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 msgr2=0x7f67b010a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.319+0000 7f67b7e00700 1 --2- 172.21.15.46:0/1768058645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b010a980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67a40098a0 tx=0x7f67a4009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.319+0000 7f67b7e00700 1 -- 172.21.15.46:0/1768058645 shutdown_connections 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.319+0000 7f67b7e00700 1 --2- 172.21.15.46:0/1768058645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b010a980 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.319+0000 7f67b7e00700 1 -- 172.21.15.46:0/1768058645 >> 172.21.15.46:0/1768058645 conn(0x7f67b007d990 msgr2=0x7f67b0108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.319+0000 7f67b7e00700 1 -- 172.21.15.46:0/1768058645 shutdown_connections 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.319+0000 7f67b7e00700 1 -- 172.21.15.46:0/1768058645 wait complete. 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.320+0000 7f67b7e00700 1 Processor -- start 2024-04-17T07:46:37.378 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.320+0000 7f67b7e00700 1 -- start start 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.320+0000 7f67b7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.320+0000 7f67b7e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010eac0 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.321+0000 7f67b5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.321+0000 7f67b5400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42928/0 (socket says 172.21.15.46:42928) 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.321+0000 7f67b5400700 1 -- 172.21.15.46:0/3395830797 learned_addr learned my addr 172.21.15.46:0/3395830797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.321+0000 7f67b5400700 1 -- 172.21.15.46:0/3395830797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b019e9c0 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.322+0000 7f67b5400700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b019e480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67a4000c00 tx=0x7f67a4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.322+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a4003b20 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.322+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67a4003cc0 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.322+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67a4024910 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.322+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b019ec50 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.322+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01a1940 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.323+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b00640b0 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.323+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f67a4022020 con 0x7f67b010a560 2024-04-17T07:46:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.324+0000 7f67ae000700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f6798039560 0x7f679803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.324+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f67a4011e00 con 0x7f67b010a560 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.324+0000 7f67b4a00700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f6798039560 0x7f679803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.325+0000 7f67b4a00700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f6798039560 0x7f679803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f679c006fd0 tx=0x7f679c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.330+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f67a401ad80 con 0x7f67b010a560 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.624+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f67b01a1b10 con 0x7f67b010a560 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.679+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f67b01a1b10 con 0x7f67b010a560 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.683+0000 7f67ae000700 1 -- 172.21.15.46:0/3395830797 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f67a401a950 con 0x7f67b010a560 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.683+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f6798039560 msgr2=0x7f679803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.683+0000 7f67b7e00700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f6798039560 0x7f679803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f679c006fd0 tx=0x7f679c006e40 comp rx=0 tx=0).stop 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.683+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 msgr2=0x7f67b019e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.683+0000 7f67b7e00700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b019e480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67a4000c00 tx=0x7f67a4011c10 comp rx=0 tx=0).stop 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.684+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 shutdown_connections 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.684+0000 7f67b7e00700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f6798039560 0x7f679803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.684+0000 7f67b7e00700 1 --2- 172.21.15.46:0/3395830797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b010a560 0x7f67b019e480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:37.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.684+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 >> 172.21.15.46:0/3395830797 conn(0x7f67b007d990 msgr2=0x7f67b0107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:37.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.685+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 shutdown_connections 2024-04-17T07:46:37.381 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:36.685+0000 7f67b7e00700 1 -- 172.21.15.46:0/3395830797 wait complete. 2024-04-17T07:46:37.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:37.491+0000 7f7788f9e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:46:37.933 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:37 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3395830797' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-17T07:46:37.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:37 smithi046 ceph-mon[32038]: mgrmap e5: x(active, since 5s) 2024-04-17T07:46:37.934 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:37.620+0000 7f7788f9e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:46:38.583 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:38.456+0000 7f7788f9e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:46:38.584 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:38.582+0000 7f7788f9e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:46:38.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:38 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3029021662' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-17T07:46:38.860 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:38.767+0000 7f7788f9e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:46:38.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.932+0000 7fe1ac400700 1 Processor -- start 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.932+0000 7fe1ac400700 1 -- start start 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.933+0000 7fe1ac400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a410b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.933+0000 7fe1ac400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a407be00 con 0x7fe1a4108e00 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.933+0000 7fe1a9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a410b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.933+0000 7fe1a9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a410b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53700/0 (socket says 172.21.15.46:53700) 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.933+0000 7fe1a9a00700 1 -- 172.21.15.46:0/2996421153 learned_addr learned my addr 172.21.15.46:0/2996421153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:38.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.934+0000 7fe1a9a00700 1 -- 172.21.15.46:0/2996421153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a410b730 con 0x7fe1a4108e00 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.934+0000 7fe1a9a00700 1 --2- 172.21.15.46:0/2996421153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a410b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098a0 tx=0x7fe198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebf7df680d353c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.934+0000 7fe1a3e00700 1 -- 172.21.15.46:0/2996421153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe198004160 con 0x7fe1a4108e00 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.935+0000 7fe1a3e00700 1 -- 172.21.15.46:0/2996421153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe198004300 con 0x7fe1a4108e00 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.935+0000 7fe1a3e00700 1 -- 172.21.15.46:0/2996421153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe198011b50 con 0x7fe1a4108e00 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.936+0000 7fe1ac400700 1 -- 172.21.15.46:0/2996421153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 msgr2=0x7fe1a410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.936+0000 7fe1ac400700 1 --2- 172.21.15.46:0/2996421153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a410b1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1980098a0 tx=0x7fe198009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.936+0000 7fe1ac400700 1 -- 172.21.15.46:0/2996421153 shutdown_connections 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.936+0000 7fe1ac400700 1 --2- 172.21.15.46:0/2996421153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a410b1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.936+0000 7fe1ac400700 1 -- 172.21.15.46:0/2996421153 >> 172.21.15.46:0/2996421153 conn(0x7fe1a4102830 msgr2=0x7fe1a4104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.936+0000 7fe1ac400700 1 -- 172.21.15.46:0/2996421153 shutdown_connections 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.936+0000 7fe1ac400700 1 -- 172.21.15.46:0/2996421153 wait complete. 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.937+0000 7fe1ac400700 1 Processor -- start 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.938+0000 7fe1ac400700 1 -- start start 2024-04-17T07:46:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.938+0000 7fe1ac400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.938+0000 7fe1ac400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a410c700 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.938+0000 7fe1a9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.938+0000 7fe1a9a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53716/0 (socket says 172.21.15.46:53716) 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.938+0000 7fe1a9a00700 1 -- 172.21.15.46:0/3029021662 learned_addr learned my addr 172.21.15.46:0/3029021662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.939+0000 7fe1a9a00700 1 -- 172.21.15.46:0/3029021662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a419c7c0 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.939+0000 7fe1a9a00700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a419c280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe198011670 tx=0x7fe198011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.939+0000 7fe1a2000700 1 -- 172.21.15.46:0/3029021662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1980039c0 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.939+0000 7fe1a2000700 1 -- 172.21.15.46:0/3029021662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe198011e50 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.939+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a419ca50 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.939+0000 7fe1a2000700 1 -- 172.21.15.46:0/3029021662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe19801a5c0 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.939+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a419f740 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.940+0000 7fe1a2000700 1 -- 172.21.15.46:0/3029021662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fe198024430 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.940+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a40640b0 con 0x7fe1a4108e00 2024-04-17T07:46:38.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.941+0000 7fe1a2000700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 0x7fe18c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.941+0000 7fe1a2000700 1 -- 172.21.15.46:0/3029021662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe19804db40 con 0x7fe1a4108e00 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.941+0000 7fe1a9000700 1 -- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 msgr2=0x7fe18c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.941+0000 7fe1a9000700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 0x7fe18c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:37.950+0000 7fe1a2000700 1 -- 172.21.15.46:0/3029021662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe19804d560 con 0x7fe1a4108e00 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.142+0000 7fe1a9000700 1 -- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 msgr2=0x7fe18c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.142+0000 7fe1a9000700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 0x7fe18c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.239+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe1a419f910 con 0x7fe1a4108e00 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.240+0000 7fe1a2000700 1 -- 172.21.15.46:0/3029021662 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fe1a419f910 con 0x7fe1a4108e00 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 msgr2=0x7fe18c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 0x7fe18c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 msgr2=0x7fe1a419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a419c280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe198011670 tx=0x7fe198011b60 comp rx=0 tx=0).stop 2024-04-17T07:46:38.922 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 shutdown_connections 2024-04-17T07:46:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7fe18c039560 0x7fe18c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 --2- 172.21.15.46:0/3029021662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1a4108e00 0x7fe1a419c280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 >> 172.21.15.46:0/3029021662 conn(0x7fe1a4102830 msgr2=0x7fe1a41032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 shutdown_connections 2024-04-17T07:46:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:38.243+0000 7fe1ac400700 1 -- 172.21.15.46:0/3029021662 wait complete. 2024-04-17T07:46:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-04-17T07:46:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-04-17T07:46:39.209 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:38.898+0000 7f7788f9e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:46:39.209 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:39.015+0000 7f7788f9e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:46:39.490 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:39.365+0000 7f7788f9e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:46:39.763 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:39.516+0000 7f7788f9e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:46:39.763 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:39.633+0000 7f7788f9e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:46:39.763 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:39.761+0000 7f7788f9e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:46:40.101 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:39.988+0000 7f7788f9e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:46:41.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:41 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:41.316+0000 7f7788f9e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:46:41.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:41 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:41.496+0000 7f7788f9e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:46:42.324 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:41 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:41.926+0000 7f7788f9e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:46:42.324 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:42 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:42.051+0000 7f7788f9e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:46:42.578 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:42 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:42.322+0000 7f7788f9e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:46:42.578 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:42 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:42.577+0000 7f7788f9e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:46:43.300 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:42 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:42.972+0000 7f7788f9e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:46:43.300 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:43 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:43.298+0000 7f7788f9e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:46:44.109 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:43 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:43.618+0000 7f7788f9e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:46:45.110 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:44 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:44.689+0000 7f7788f9e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:46:45.708 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:45.414+0000 7f7788f9e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:46:45.708 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:45.549+0000 7f7788f9e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:46:46.110 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:46:45.846+0000 7f7788f9e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:46:46.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: Active manager daemon x restarted 2024-04-17T07:46:46.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: Activating manager daemon x 2024-04-17T07:46:46.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: osdmap e2: 0 total, 0 up, 0 in 2024-04-17T07:46:46.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: mgrmap e6: x(active, starting, since 0.0104907s) 2024-04-17T07:46:46.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:46:46.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T07:46:46.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T07:46:46.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T07:46:46.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T07:46:46.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:45 smithi046 ceph-mon[32038]: Manager daemon x is now available 2024-04-17T07:46:47.224 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: Found migration_current of "None". Setting to last migration. 2024-04-17T07:46:47.224 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:46:47.224 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:46:47.225 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:46:47.225 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:46:47.225 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T07:46:47.225 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T07:46:47.225 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:46:47.225 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:46:47.225 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:46 smithi046 ceph-mon[32038]: mgrmap e7: x(active, since 1.01682s) 2024-04-17T07:46:47.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:46:47.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-17T07:46:47.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-17T07:46:47.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:46:47.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.489+0000 7f4fa7200700 1 Processor -- start 2024-04-17T07:46:47.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.489+0000 7f4fa7200700 1 -- start start 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.490+0000 7f4fa7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.490+0000 7f4fa7200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa807c7f0 con 0x7f4fa807ddb0 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.490+0000 7f4fa5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.490+0000 7f4fa5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53718/0 (socket says 172.21.15.46:53718) 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.490+0000 7f4fa5e00700 1 -- 172.21.15.46:0/459237656 learned_addr learned my addr 172.21.15.46:0/459237656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.491+0000 7f4fa5e00700 1 -- 172.21.15.46:0/459237656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa807c970 con 0x7f4fa807ddb0 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.491+0000 7f4fa5e00700 1 --2- 172.21.15.46:0/459237656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa807c2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4f980098a0 tx=0x7f4f98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e097bd1691a4f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.491+0000 7f4fa4a00700 1 -- 172.21.15.46:0/459237656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98004160 con 0x7f4fa807ddb0 2024-04-17T07:46:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.491+0000 7f4fa4a00700 1 -- 172.21.15.46:0/459237656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f98004300 con 0x7f4fa807ddb0 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.491+0000 7f4fa4a00700 1 -- 172.21.15.46:0/459237656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98011b50 con 0x7f4fa807ddb0 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.492+0000 7f4fa7200700 1 -- 172.21.15.46:0/459237656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 msgr2=0x7f4fa807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.492+0000 7f4fa7200700 1 --2- 172.21.15.46:0/459237656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa807c2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4f980098a0 tx=0x7f4f98009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.493+0000 7f4fa7200700 1 -- 172.21.15.46:0/459237656 shutdown_connections 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.493+0000 7f4fa7200700 1 --2- 172.21.15.46:0/459237656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa807c2b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.493+0000 7f4fa7200700 1 -- 172.21.15.46:0/459237656 >> 172.21.15.46:0/459237656 conn(0x7f4fa8103760 msgr2=0x7f4fa8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.493+0000 7f4fa7200700 1 -- 172.21.15.46:0/459237656 shutdown_connections 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.493+0000 7f4fa7200700 1 -- 172.21.15.46:0/459237656 wait complete. 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.494+0000 7f4fa7200700 1 Processor -- start 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.494+0000 7f4fa7200700 1 -- start start 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.495+0000 7f4fa7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.495+0000 7f4fa7200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa810e750 con 0x7f4fa807ddb0 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.495+0000 7f4fa5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:47.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.495+0000 7f4fa5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53726/0 (socket says 172.21.15.46:53726) 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.495+0000 7f4fa5e00700 1 -- 172.21.15.46:0/3252924900 learned_addr learned my addr 172.21.15.46:0/3252924900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.495+0000 7f4fa5e00700 1 -- 172.21.15.46:0/3252924900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa819e880 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.496+0000 7f4fa5e00700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa819e340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4f98011b60 tx=0x7f4f98011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.496+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98003970 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.496+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f98003b10 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.496+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa819eb10 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.496+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f98024610 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.496+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa81a1800 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.497+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f4f980248c0 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.498+0000 7f4f9ea00700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.498+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 --> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f8c03c1d0 con 0x7f4f8c039600 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.498+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4f98011e00 con 0x7f4fa807ddb0 2024-04-17T07:46:47.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.498+0000 7f4fa5400700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 msgr2=0x7f4f8c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.498+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.699+0000 7f4fa5400700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 msgr2=0x7f4f8c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:39.699+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:40.100+0000 7f4fa5400700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 msgr2=0x7f4f8c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:40.100+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:40.901+0000 7f4fa5400700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 msgr2=0x7f4f8c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:40.901+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:42.503+0000 7f4fa5400700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 msgr2=0x7f4f8c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:42.503+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:45.706+0000 7f4fa5400700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 msgr2=0x7f4f8c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/4094803685 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:45.707+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:45.857+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f4f98024bb0 con 0x7f4fa807ddb0 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:45.857+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 msgr2=0x7f4f8c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:45.857+0000 7f4f9ea00700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:47.551 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.862+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f4f9804ef20 con 0x7f4fa807ddb0 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.862+0000 7f4f9ea00700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4f8c03cef0 0x7f4f8c03f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.862+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f98012ea0 con 0x7f4f8c03cef0 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.864+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4f8c03cef0 0x7f4f8c03f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.866+0000 7f4fa5400700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4f8c03cef0 0x7f4f8c03f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f94003a10 tx=0x7f4f940092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.868+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4f98012ea0 con 0x7f4f8c03cef0 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.876+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4fa80640b0 con 0x7f4f8c03cef0 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.876+0000 7f4f9ea00700 1 -- 172.21.15.46:0/3252924900 <== mgr.14120 v2:172.21.15.46:6800/461113542 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f4fa80640b0 con 0x7f4f8c03cef0 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.877+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4f8c03cef0 msgr2=0x7f4f8c03f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.877+0000 7f4fa7200700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4f8c03cef0 0x7f4f8c03f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f94003a10 tx=0x7f4f940092b0 comp rx=0 tx=0).stop 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.877+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 msgr2=0x7f4fa819e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.877+0000 7f4fa7200700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa819e340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4f98011b60 tx=0x7f4f98011c40 comp rx=0 tx=0).stop 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.878+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 shutdown_connections 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.878+0000 7f4fa7200700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4f8c03cef0 0x7f4f8c03f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.878+0000 7f4fa7200700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa807ddb0 0x7f4fa819e340 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.878+0000 7f4fa7200700 1 --2- 172.21.15.46:0/3252924900 >> [v2:172.21.15.46:6800/4094803685,v1:172.21.15.46:6801/4094803685] conn(0x7f4f8c039600 0x7f4f8c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.878+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 >> 172.21.15.46:0/3252924900 conn(0x7f4fa8103760 msgr2=0x7f4fa81042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.878+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 shutdown_connections 2024-04-17T07:46:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:46.878+0000 7f4fa7200700 1 -- 172.21.15.46:0/3252924900 wait complete. 2024-04-17T07:46:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-04-17T07:46:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-04-17T07:46:48.096 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:47 smithi046 ceph-mon[32038]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-17T07:46:48.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:47 smithi046 ceph-mon[32038]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-17T07:46:48.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:47 smithi046 ceph-mon[32038]: [17/Apr/2024:07:46:47] ENGINE Bus STARTING 2024-04-17T07:46:48.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:47 smithi046 ceph-mon[32038]: [17/Apr/2024:07:46:47] ENGINE Serving on http://172.21.15.46:8765 2024-04-17T07:46:48.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:47 smithi046 ceph-mon[32038]: [17/Apr/2024:07:46:47] ENGINE Serving on https://172.21.15.46:7150 2024-04-17T07:46:48.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:47 smithi046 ceph-mon[32038]: [17/Apr/2024:07:46:47] ENGINE Bus STARTED 2024-04-17T07:46:48.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:47 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:46:49.092 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:48 smithi046 ceph-mon[32038]: mgrmap e8: x(active, since 2s) 2024-04-17T07:46:49.092 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:48 smithi046 ceph-mon[32038]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:46:49.092 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:48 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:46:49.092 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:48 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:46:49.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.187+0000 7f0b88400700 1 Processor -- start 2024-04-17T07:46:49.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.188+0000 7f0b88400700 1 -- start start 2024-04-17T07:46:49.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.188+0000 7f0b88400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8010a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:49.101 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.188+0000 7f0b88400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8010aef0 con 0x7f0b8010a500 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.188+0000 7f0b85a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8010a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.188+0000 7f0b85a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8010a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43716/0 (socket says 172.21.15.46:43716) 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.188+0000 7f0b85a00700 1 -- 172.21.15.46:0/3027113090 learned_addr learned my addr 172.21.15.46:0/3027113090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.189+0000 7f0b85a00700 1 -- 172.21.15.46:0/3027113090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8010b700 con 0x7f0b8010a500 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.189+0000 7f0b85a00700 1 --2- 172.21.15.46:0/3027113090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8010a920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b740098a0 tx=0x7f0b74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186836e36236a2f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.189+0000 7f0b7fe00700 1 -- 172.21.15.46:0/3027113090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b74004160 con 0x7f0b8010a500 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.189+0000 7f0b7fe00700 1 -- 172.21.15.46:0/3027113090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b74004300 con 0x7f0b8010a500 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.189+0000 7f0b7fe00700 1 -- 172.21.15.46:0/3027113090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b74011b50 con 0x7f0b8010a500 2024-04-17T07:46:49.103 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.190+0000 7f0b88400700 1 -- 172.21.15.46:0/3027113090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 msgr2=0x7f0b8010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:49.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.190+0000 7f0b88400700 1 --2- 172.21.15.46:0/3027113090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8010a920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b740098a0 tx=0x7f0b74009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:49.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.190+0000 7f0b88400700 1 -- 172.21.15.46:0/3027113090 shutdown_connections 2024-04-17T07:46:49.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.190+0000 7f0b88400700 1 --2- 172.21.15.46:0/3027113090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8010a920 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:49.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.190+0000 7f0b88400700 1 -- 172.21.15.46:0/3027113090 >> 172.21.15.46:0/3027113090 conn(0x7f0b8007d9c0 msgr2=0x7f0b80108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:49.104 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.190+0000 7f0b88400700 1 -- 172.21.15.46:0/3027113090 shutdown_connections 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.190+0000 7f0b88400700 1 -- 172.21.15.46:0/3027113090 wait complete. 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.191+0000 7f0b88400700 1 Processor -- start 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.192+0000 7f0b88400700 1 -- start start 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.192+0000 7f0b88400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.192+0000 7f0b88400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8010eab0 con 0x7f0b8010a500 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.192+0000 7f0b85a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.192+0000 7f0b85a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43722/0 (socket says 172.21.15.46:43722) 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.192+0000 7f0b85a00700 1 -- 172.21.15.46:0/1289708808 learned_addr learned my addr 172.21.15.46:0/1289708808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.192+0000 7f0b85a00700 1 -- 172.21.15.46:0/1289708808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8019ea30 con 0x7f0b8010a500 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.193+0000 7f0b85a00700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8019e4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009fb0 tx=0x7f0b74011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.193+0000 7f0b7e000700 1 -- 172.21.15.46:0/1289708808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b740038b0 con 0x7f0b8010a500 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.193+0000 7f0b7e000700 1 -- 172.21.15.46:0/1289708808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b74011e50 con 0x7f0b8010a500 2024-04-17T07:46:49.108 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.193+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8019ecc0 con 0x7f0b8010a500 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.193+0000 7f0b7e000700 1 -- 172.21.15.46:0/1289708808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b74019550 con 0x7f0b8010a500 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.193+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801a19b0 con 0x7f0b8010a500 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.194+0000 7f0b7e000700 1 -- 172.21.15.46:0/1289708808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0b74003a50 con 0x7f0b8010a500 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.194+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b800640b0 con 0x7f0b8010a500 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.195+0000 7f0b7e000700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f0b68039560 0x7f0b6803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.195+0000 7f0b7e000700 1 -- 172.21.15.46:0/1289708808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b7404da00 con 0x7f0b8010a500 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.195+0000 7f0b85000700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f0b68039560 0x7f0b6803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.196+0000 7f0b85000700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f0b68039560 0x7f0b6803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c006fd0 tx=0x7f0b6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.202+0000 7f0b7e000700 1 -- 172.21.15.46:0/1289708808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0b74020240 con 0x7f0b8010a500 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.451+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0b801082c0 con 0x7f0b68039560 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.461+0000 7f0b7e000700 1 -- 172.21.15.46:0/1289708808 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b801082c0 con 0x7f0b68039560 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f0b68039560 msgr2=0x7f0b6803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f0b68039560 0x7f0b6803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c006fd0 tx=0x7f0b6c006e40 comp rx=0 tx=0).stop 2024-04-17T07:46:49.109 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 msgr2=0x7f0b8019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8019e4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009fb0 tx=0x7f0b74011bf0 comp rx=0 tx=0).stop 2024-04-17T07:46:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 shutdown_connections 2024-04-17T07:46:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f0b68039560 0x7f0b6803ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 --2- 172.21.15.46:0/1289708808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8010a500 0x7f0b8019e4f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 >> 172.21.15.46:0/1289708808 conn(0x7f0b8007d9c0 msgr2=0x7f0b80107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 shutdown_connections 2024-04-17T07:46:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:48.464+0000 7f0b88400700 1 -- 172.21.15.46:0/1289708808 wait complete. 2024-04-17T07:46:50.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-17T07:46:50.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.689+0000 7f6fdd800700 1 Processor -- start 2024-04-17T07:46:50.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.689+0000 7f6fdd800700 1 -- start start 2024-04-17T07:46:50.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.690+0000 7f6fdd800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd810a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:50.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.690+0000 7f6fdd800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd810af50 con 0x7f6fd810a560 2024-04-17T07:46:50.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.690+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd810a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:50.513 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.690+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd810a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43738/0 (socket says 172.21.15.46:43738) 2024-04-17T07:46:50.516 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.690+0000 7f6fd6a00700 1 -- 172.21.15.46:0/1518891098 learned_addr learned my addr 172.21.15.46:0/1518891098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:50.516 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.691+0000 7f6fd6a00700 1 -- 172.21.15.46:0/1518891098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd810b710 con 0x7f6fd810a560 2024-04-17T07:46:50.516 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.691+0000 7f6fd6a00700 1 --2- 172.21.15.46:0/1518891098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd810a980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098a0 tx=0x7f6fc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a5d052d38bc678 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:50.516 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.692+0000 7f6fd5600700 1 -- 172.21.15.46:0/1518891098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc8004160 con 0x7f6fd810a560 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.692+0000 7f6fd5600700 1 -- 172.21.15.46:0/1518891098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fc8004300 con 0x7f6fd810a560 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.692+0000 7f6fd5600700 1 -- 172.21.15.46:0/1518891098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc8011b50 con 0x7f6fd810a560 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.692+0000 7f6fdd800700 1 -- 172.21.15.46:0/1518891098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 msgr2=0x7f6fd810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.692+0000 7f6fdd800700 1 --2- 172.21.15.46:0/1518891098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd810a980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098a0 tx=0x7f6fc8009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.693+0000 7f6fdd800700 1 -- 172.21.15.46:0/1518891098 shutdown_connections 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.693+0000 7f6fdd800700 1 --2- 172.21.15.46:0/1518891098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd810a980 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.693+0000 7f6fdd800700 1 -- 172.21.15.46:0/1518891098 >> 172.21.15.46:0/1518891098 conn(0x7f6fd807d990 msgr2=0x7f6fd8108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.693+0000 7f6fdd800700 1 -- 172.21.15.46:0/1518891098 shutdown_connections 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.693+0000 7f6fdd800700 1 -- 172.21.15.46:0/1518891098 wait complete. 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.694+0000 7f6fdd800700 1 Processor -- start 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.694+0000 7f6fdd800700 1 -- start start 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.695+0000 7f6fdd800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.695+0000 7f6fdd800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd810eac0 con 0x7f6fd810a560 2024-04-17T07:46:50.517 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.695+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:50.518 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.695+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43748/0 (socket says 172.21.15.46:43748) 2024-04-17T07:46:50.518 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.695+0000 7f6fd6a00700 1 -- 172.21.15.46:0/2468270375 learned_addr learned my addr 172.21.15.46:0/2468270375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:50.518 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.696+0000 7f6fd6a00700 1 -- 172.21.15.46:0/2468270375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd819e9b0 con 0x7f6fd810a560 2024-04-17T07:46:50.518 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.696+0000 7f6fd6a00700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd819e470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8011670 tx=0x7f6fc8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:50.518 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.696+0000 7f6fcf400700 1 -- 172.21.15.46:0/2468270375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc80039c0 con 0x7f6fd810a560 2024-04-17T07:46:50.518 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.696+0000 7f6fcf400700 1 -- 172.21.15.46:0/2468270375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fc8011e50 con 0x7f6fd810a560 2024-04-17T07:46:50.518 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.696+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd819ec40 con 0x7f6fd810a560 2024-04-17T07:46:50.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.696+0000 7f6fcf400700 1 -- 172.21.15.46:0/2468270375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc801a5c0 con 0x7f6fd810a560 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.697+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd81a1930 con 0x7f6fd810a560 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.698+0000 7f6fcf400700 1 -- 172.21.15.46:0/2468270375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6fc801a870 con 0x7f6fd810a560 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.698+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd80640b0 con 0x7f6fd810a560 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.698+0000 7f6fcf400700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f6fbc0395b0 0x7f6fbc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.698+0000 7f6fcf400700 1 -- 172.21.15.46:0/2468270375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6fc804dcd0 con 0x7f6fd810a560 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.698+0000 7f6fd6000700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f6fbc0395b0 0x7f6fbc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.699+0000 7f6fd6000700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f6fbc0395b0 0x7f6fbc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4006fd0 tx=0x7f6fc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.705+0000 7f6fcf400700 1 -- 172.21.15.46:0/2468270375 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6fc8019c70 con 0x7f6fd810a560 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.938+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6fd8108220 con 0x7f6fbc0395b0 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.940+0000 7f6fcf400700 1 -- 172.21.15.46:0/2468270375 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6fd8108220 con 0x7f6fbc0395b0 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.942+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f6fbc0395b0 msgr2=0x7f6fbc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:50.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.942+0000 7f6fdd800700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f6fbc0395b0 0x7f6fbc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4006fd0 tx=0x7f6fc4006e40 comp rx=0 tx=0).stop 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.942+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 msgr2=0x7f6fd819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.942+0000 7f6fdd800700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd819e470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8011670 tx=0x7f6fc8011b60 comp rx=0 tx=0).stop 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.943+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 shutdown_connections 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.943+0000 7f6fdd800700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f6fbc0395b0 0x7f6fbc03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.943+0000 7f6fdd800700 1 --2- 172.21.15.46:0/2468270375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd810a560 0x7f6fd819e470 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.943+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 >> 172.21.15.46:0/2468270375 conn(0x7f6fd807d990 msgr2=0x7f6fd8107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.943+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 shutdown_connections 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:49.943+0000 7f6fdd800700 1 -- 172.21.15.46:0/2468270375 wait complete. 2024-04-17T07:46:50.522 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-04-17T07:46:51.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:50 smithi046 ceph-mon[32038]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:46:52.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-mon[32038]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:46:52.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-mon[32038]: Generating ssh key... 2024-04-17T07:46:52.110 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: Generating public/private rsa key pair. 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: Your identification has been saved in /tmp/tmpg3aqvcaj/key. 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: Your public key has been saved in /tmp/tmpg3aqvcaj/key.pub. 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: The key fingerprint is: 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: SHA256:UywHMLtRs+K/iXnCUzrTK9AoOjJ8eD0j/XJEMfnpxGs ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: The key's randomart image is: 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: +---[RSA 3072]----+ 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: | o.= | 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: | B = | 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: | + B = | 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: | . = O | 2024-04-17T07:46:52.111 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: | o+ S . | 2024-04-17T07:46:52.112 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: | . o .o.E | 2024-04-17T07:46:52.112 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: |....oo.+o | 2024-04-17T07:46:52.112 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: |=o + *O+oo | 2024-04-17T07:46:52.112 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: |.oo . OO+. | 2024-04-17T07:46:52.112 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:46:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: +----[SHA256]-----+ 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.117+0000 7f080fe00700 1 Processor -- start 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.117+0000 7f080fe00700 1 -- start start 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.117+0000 7f080fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f0808108720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.117+0000 7f080fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0808108cf0 con 0x7f0808108300 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.118+0000 7f080d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f0808108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.118+0000 7f080d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f0808108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43752/0 (socket says 172.21.15.46:43752) 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.118+0000 7f080d400700 1 -- 172.21.15.46:0/1406505415 learned_addr learned my addr 172.21.15.46:0/1406505415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.118+0000 7f080d400700 1 -- 172.21.15.46:0/1406505415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0808109500 con 0x7f0808108300 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.119+0000 7f080d400700 1 --2- 172.21.15.46:0/1406505415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f0808108720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098a0 tx=0x7f07fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b57980eba21ff09 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:52.664 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.119+0000 7f0807e00700 1 -- 172.21.15.46:0/1406505415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc004160 con 0x7f0808108300 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.119+0000 7f0807e00700 1 -- 172.21.15.46:0/1406505415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07fc004300 con 0x7f0808108300 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.119+0000 7f0807e00700 1 -- 172.21.15.46:0/1406505415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc011b50 con 0x7f0808108300 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.120+0000 7f080fe00700 1 -- 172.21.15.46:0/1406505415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 msgr2=0x7f0808108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.120+0000 7f080fe00700 1 --2- 172.21.15.46:0/1406505415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f0808108720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0098a0 tx=0x7f07fc009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.120+0000 7f080fe00700 1 -- 172.21.15.46:0/1406505415 shutdown_connections 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.120+0000 7f080fe00700 1 --2- 172.21.15.46:0/1406505415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f0808108720 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.120+0000 7f080fe00700 1 -- 172.21.15.46:0/1406505415 >> 172.21.15.46:0/1406505415 conn(0x7f08081038a0 msgr2=0x7f0808105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.120+0000 7f080fe00700 1 -- 172.21.15.46:0/1406505415 shutdown_connections 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.120+0000 7f080fe00700 1 -- 172.21.15.46:0/1406505415 wait complete. 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.121+0000 7f080fe00700 1 Processor -- start 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.121+0000 7f080fe00700 1 -- start start 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.122+0000 7f080fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f080807db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.122+0000 7f080fe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f080810c8b0 con 0x7f0808108300 2024-04-17T07:46:52.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.122+0000 7f080d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f080807db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.122+0000 7f080d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f080807db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43762/0 (socket says 172.21.15.46:43762) 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.122+0000 7f080d400700 1 -- 172.21.15.46:0/3204276060 learned_addr learned my addr 172.21.15.46:0/3204276060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.122+0000 7f080d400700 1 -- 172.21.15.46:0/3204276060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080807e060 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.123+0000 7f080d400700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f080807db20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009fb0 tx=0x7f07fc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.123+0000 7f0806000700 1 -- 172.21.15.46:0/3204276060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc0038b0 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.123+0000 7f0806000700 1 -- 172.21.15.46:0/3204276060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07fc011e50 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.123+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080807c250 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.123+0000 7f0806000700 1 -- 172.21.15.46:0/3204276060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07fc019550 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.123+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080807c790 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.124+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08080640b0 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.124+0000 7f0806000700 1 -- 172.21.15.46:0/3204276060 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f07fc01a430 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.125+0000 7f0806000700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f07f0039560 0x7f07f003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.125+0000 7f0806000700 1 -- 172.21.15.46:0/3204276060 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f07fc04da20 con 0x7f0808108300 2024-04-17T07:46:52.666 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.126+0000 7f080ca00700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f07f0039560 0x7f07f003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.127+0000 7f080ca00700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f07f0039560 0x7f07f003ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f07f4006fd0 tx=0x7f07f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.131+0000 7f0806000700 1 -- 172.21.15.46:0/3204276060 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f07fc020240 con 0x7f0808108300 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.364+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0808104b50 con 0x7f07f0039560 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.976+0000 7f0806000700 1 -- 172.21.15.46:0/3204276060 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0808104b50 con 0x7f07f0039560 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.978+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f07f0039560 msgr2=0x7f07f003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.978+0000 7f080fe00700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f07f0039560 0x7f07f003ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f07f4006fd0 tx=0x7f07f4006e40 comp rx=0 tx=0).stop 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.978+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 msgr2=0x7f080807db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.978+0000 7f080fe00700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f080807db20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009fb0 tx=0x7f07fc011bf0 comp rx=0 tx=0).stop 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.979+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 shutdown_connections 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.979+0000 7f080fe00700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f07f0039560 0x7f07f003ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.979+0000 7f080fe00700 1 --2- 172.21.15.46:0/3204276060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808108300 0x7f080807db20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.979+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 >> 172.21.15.46:0/3204276060 conn(0x7f08081038a0 msgr2=0x7f0808104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.979+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 shutdown_connections 2024-04-17T07:46:52.667 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:51.979+0000 7f080fe00700 1 -- 172.21.15.46:0/3204276060 wait complete. 2024-04-17T07:46:53.107 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:52 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:46:53.107 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:52 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:46:54.228 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:53 smithi046 ceph-mon[32038]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:46:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2nUp2BhXlccuNtfzvbeZ5qNHV5sASbI0+ojkcm9ZfRE7R1JfVKa7HIMulm8U7qWVuQQYgRV+VjWSPolO41/ca6ZkhTUumhvGAjclMQ95LXJwtHYPSJ+/w25dfREDxtyj6FWO2i+01EuqEeusWWeG1vWGAp4ee/JUl2Nc7xDbRqqqgKxthyZHyvR6XmpoQ3F3DwRI3Ojli031Z+hF6rZZ3n3kugfc/1+FF9EwUr4z/nL5kZCmyURZvrKfza7oe/VYjX5XecpqJ2kFlZyBMsJQOobRuZXgs5PrF42HLwUwNXIpOv/cJVCnFJsTPXnE4wysma8XFcCpvOKejXaV71yfHWFxeQYYYABDLJB1M4l9JFgWDcmdDkj3GGq/ZEgpKuI57WAZ/0fuqQCYVCZ0U7qN7N6j2YOmjNT0p6qcWa4+HAGow+52K86no6TQLXBKakMGR2Qu5ZopDHsojeG9nFtWDWLzsuRpTsuUj26T32DS7DnXcKScL8v/RYiCQDx3rsZk= ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:46:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.264+0000 7f4976800700 1 Processor -- start 2024-04-17T07:46:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.264+0000 7f4976800700 1 -- start start 2024-04-17T07:46:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.264+0000 7f4976800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497007c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.264+0000 7f4976800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f497007c7d0 con 0x7f497007dd90 2024-04-17T07:46:54.245 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.265+0000 7f4975400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497007c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.265+0000 7f4975400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497007c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43770/0 (socket says 172.21.15.46:43770) 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.265+0000 7f4975400700 1 -- 172.21.15.46:0/4211079799 learned_addr learned my addr 172.21.15.46:0/4211079799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.265+0000 7f4975400700 1 -- 172.21.15.46:0/4211079799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497007c950 con 0x7f497007dd90 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.266+0000 7f4975400700 1 --2- 172.21.15.46:0/4211079799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497007c290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49640098a0 tx=0x7f4964009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383f202b8fb64c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.266+0000 7f496fe00700 1 -- 172.21.15.46:0/4211079799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4964004160 con 0x7f497007dd90 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.266+0000 7f496fe00700 1 -- 172.21.15.46:0/4211079799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4964004300 con 0x7f497007dd90 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.266+0000 7f496fe00700 1 -- 172.21.15.46:0/4211079799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4964011b50 con 0x7f497007dd90 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.267+0000 7f4976800700 1 -- 172.21.15.46:0/4211079799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 msgr2=0x7f497007c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.267+0000 7f4976800700 1 --2- 172.21.15.46:0/4211079799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497007c290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49640098a0 tx=0x7f4964009bb0 comp rx=0 tx=0).stop 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.267+0000 7f4976800700 1 -- 172.21.15.46:0/4211079799 shutdown_connections 2024-04-17T07:46:54.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.267+0000 7f4976800700 1 --2- 172.21.15.46:0/4211079799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497007c290 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:54.249 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.267+0000 7f4976800700 1 -- 172.21.15.46:0/4211079799 >> 172.21.15.46:0/4211079799 conn(0x7f4970103760 msgr2=0x7f4970105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:54.249 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.267+0000 7f4976800700 1 -- 172.21.15.46:0/4211079799 shutdown_connections 2024-04-17T07:46:54.249 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.267+0000 7f4976800700 1 -- 172.21.15.46:0/4211079799 wait complete. 2024-04-17T07:46:54.249 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.268+0000 7f4976800700 1 Processor -- start 2024-04-17T07:46:54.249 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.268+0000 7f4976800700 1 -- start start 2024-04-17T07:46:54.249 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.268+0000 7f4976800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:54.249 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.268+0000 7f4976800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f497010e710 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.269+0000 7f4975400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.269+0000 7f4975400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43782/0 (socket says 172.21.15.46:43782) 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.269+0000 7f4975400700 1 -- 172.21.15.46:0/3526632001 learned_addr learned my addr 172.21.15.46:0/3526632001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.269+0000 7f4975400700 1 -- 172.21.15.46:0/3526632001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497019e860 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.269+0000 7f4975400700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497019e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4964011b60 tx=0x7f4964011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.270+0000 7f496e000700 1 -- 172.21.15.46:0/3526632001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4964003970 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.270+0000 7f496e000700 1 -- 172.21.15.46:0/3526632001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4964003b10 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.270+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497019eaf0 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.270+0000 7f496e000700 1 -- 172.21.15.46:0/3526632001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4964024610 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.270+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49701a17e0 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.271+0000 7f496e000700 1 -- 172.21.15.46:0/3526632001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f496401a9f0 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.271+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49700640b0 con 0x7f497007dd90 2024-04-17T07:46:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.271+0000 7f496e000700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4958039600 0x7f495803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:46:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.272+0000 7f496e000700 1 -- 172.21.15.46:0/3526632001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4964011e00 con 0x7f497007dd90 2024-04-17T07:46:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.272+0000 7f4974a00700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4958039600 0x7f495803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:46:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.273+0000 7f4974a00700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4958039600 0x7f495803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f495c006fd0 tx=0x7f495c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:46:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.277+0000 7f496e000700 1 -- 172.21.15.46:0/3526632001 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f496400b3e0 con 0x7f497007dd90 2024-04-17T07:46:54.252 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.505+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4970104a50 con 0x7f4958039600 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.506+0000 7f496e000700 1 -- 172.21.15.46:0/3526632001 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4970104a50 con 0x7f4958039600 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4958039600 msgr2=0x7f495803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4958039600 0x7f495803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f495c006fd0 tx=0x7f495c006e40 comp rx=0 tx=0).stop 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 msgr2=0x7f497019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497019e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4964011b60 tx=0x7f4964011c40 comp rx=0 tx=0).stop 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 shutdown_connections 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f4958039600 0x7f495803bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 --2- 172.21.15.46:0/3526632001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f497007dd90 0x7f497019e320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.508+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 >> 172.21.15.46:0/3526632001 conn(0x7f4970103760 msgr2=0x7f4970104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.509+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 shutdown_connections 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:53.509+0000 7f4976800700 1 -- 172.21.15.46:0/3526632001 wait complete. 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-17T07:46:54.253 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-04-17T07:46:54.254 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-04-17T07:46:55.952 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:55 smithi046 ceph-mon[32038]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:46:57.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:46:56 smithi046 ceph-mon[32038]: Deploying cephadm binary to smithi046 2024-04-17T07:47:02.755 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-04-17T07:47:02.755 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.871+0000 7fb0fb000700 1 Processor -- start 2024-04-17T07:47:02.755 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.871+0000 7fb0fb000700 1 -- start start 2024-04-17T07:47:02.755 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.872+0000 7fb0fb000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:02.755 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.872+0000 7fb0fb000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f407c8a0 con 0x7fb0f407de70 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.872+0000 7fb0f3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.872+0000 7fb0f3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43792/0 (socket says 172.21.15.46:43792) 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.872+0000 7fb0f3e00700 1 -- 172.21.15.46:0/666873983 learned_addr learned my addr 172.21.15.46:0/666873983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.873+0000 7fb0f3e00700 1 -- 172.21.15.46:0/666873983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f407ca20 con 0x7fb0f407de70 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.873+0000 7fb0f3e00700 1 --2- 172.21.15.46:0/666873983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f407c2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098a0 tx=0x7fb0e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f948da4456c3f724 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.873+0000 7fb0f3400700 1 -- 172.21.15.46:0/666873983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e4004160 con 0x7fb0f407de70 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.873+0000 7fb0f3400700 1 -- 172.21.15.46:0/666873983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0e4004300 con 0x7fb0f407de70 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.873+0000 7fb0f3400700 1 -- 172.21.15.46:0/666873983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e4011b50 con 0x7fb0f407de70 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.874+0000 7fb0fb000700 1 -- 172.21.15.46:0/666873983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 msgr2=0x7fb0f407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.874+0000 7fb0fb000700 1 --2- 172.21.15.46:0/666873983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f407c2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098a0 tx=0x7fb0e4009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.874+0000 7fb0fb000700 1 -- 172.21.15.46:0/666873983 shutdown_connections 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.874+0000 7fb0fb000700 1 --2- 172.21.15.46:0/666873983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f407c2d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:02.756 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.874+0000 7fb0fb000700 1 -- 172.21.15.46:0/666873983 >> 172.21.15.46:0/666873983 conn(0x7fb0f41038a0 msgr2=0x7fb0f4105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.874+0000 7fb0fb000700 1 -- 172.21.15.46:0/666873983 shutdown_connections 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.874+0000 7fb0fb000700 1 -- 172.21.15.46:0/666873983 wait complete. 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.876+0000 7fb0fb000700 1 Processor -- start 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.876+0000 7fb0fb000700 1 -- start start 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.876+0000 7fb0fb000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.876+0000 7fb0fb000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f410e8f0 con 0x7fb0f407de70 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.876+0000 7fb0f3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.876+0000 7fb0f3e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f419e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43808/0 (socket says 172.21.15.46:43808) 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.876+0000 7fb0f3e00700 1 -- 172.21.15.46:0/2121181731 learned_addr learned my addr 172.21.15.46:0/2121181731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.877+0000 7fb0f3e00700 1 -- 172.21.15.46:0/2121181731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f419ea10 con 0x7fb0f407de70 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.877+0000 7fb0f3e00700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f419e4d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4011850 tx=0x7fb0e4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.877+0000 7fb0f1600700 1 -- 172.21.15.46:0/2121181731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e4003af0 con 0x7fb0f407de70 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.877+0000 7fb0f1600700 1 -- 172.21.15.46:0/2121181731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0e4020070 con 0x7fb0f407de70 2024-04-17T07:47:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.877+0000 7fb0f1600700 1 -- 172.21.15.46:0/2121181731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0e40195a0 con 0x7fb0f407de70 2024-04-17T07:47:02.758 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.877+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f419eca0 con 0x7fb0f407de70 2024-04-17T07:47:02.758 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.878+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f41a1990 con 0x7fb0f407de70 2024-04-17T07:47:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.879+0000 7fb0f1600700 1 -- 172.21.15.46:0/2121181731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb0e4019850 con 0x7fb0f407de70 2024-04-17T07:47:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.879+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f40640b0 con 0x7fb0f407de70 2024-04-17T07:47:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.879+0000 7fb0f1600700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fb0d8039600 0x7fb0d803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.879+0000 7fb0f1600700 1 -- 172.21.15.46:0/2121181731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb0e4011e00 con 0x7fb0f407de70 2024-04-17T07:47:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.880+0000 7fb0ebe00700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fb0d8039600 0x7fb0d803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.880+0000 7fb0ebe00700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fb0d8039600 0x7fb0d803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc006fd0 tx=0x7fb0dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:54.885+0000 7fb0f1600700 1 -- 172.21.15.46:0/2121181731 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb0e400e260 con 0x7fb0f407de70 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:46:55.126+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}) -- 0x7fb0f4104ae0 con 0x7fb0d8039600 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.928+0000 7fb0f1600700 1 -- 172.21.15.46:0/2121181731 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb0f4104ae0 con 0x7fb0d8039600 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fb0d8039600 msgr2=0x7fb0d803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fb0d8039600 0x7fb0d803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc006fd0 tx=0x7fb0dc006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 msgr2=0x7fb0f419e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f419e4d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4011850 tx=0x7fb0e4011c10 comp rx=0 tx=0).stop 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 shutdown_connections 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fb0d8039600 0x7fb0d803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 --2- 172.21.15.46:0/2121181731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0f407de70 0x7fb0f419e4d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 >> 172.21.15.46:0/2121181731 conn(0x7fb0f41038a0 msgr2=0x7fb0f41043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 shutdown_connections 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:01.932+0000 7fb0fb000700 1 -- 172.21.15.46:0/2121181731 wait complete. 2024-04-17T07:47:02.761 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-04-17T07:47:03.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:02 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:47:03.077 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:02 smithi046 ceph-mon[32038]: Added host smithi046 2024-04-17T07:47:03.077 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:02 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:47:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-17T07:47:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.454+0000 7f2c1ee00700 1 Processor -- start 2024-04-17T07:47:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.455+0000 7f2c1ee00700 1 -- start start 2024-04-17T07:47:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.455+0000 7f2c1ee00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:04.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.455+0000 7f2c1ee00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18110890 con 0x7f2c1810fea0 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.456+0000 7f2c17e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.456+0000 7f2c17e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52882/0 (socket says 172.21.15.46:52882) 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.456+0000 7f2c17e00700 1 -- 172.21.15.46:0/1512262263 learned_addr learned my addr 172.21.15.46:0/1512262263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.456+0000 7f2c17e00700 1 -- 172.21.15.46:0/1512262263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c18110a10 con 0x7f2c1810fea0 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.456+0000 7f2c17e00700 1 --2- 172.21.15.46:0/1512262263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181102c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c080098a0 tx=0x7f2c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af8a79f49a7d265 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.457+0000 7f2c17400700 1 -- 172.21.15.46:0/1512262263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c08004160 con 0x7f2c1810fea0 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.457+0000 7f2c17400700 1 -- 172.21.15.46:0/1512262263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c08004300 con 0x7f2c1810fea0 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.457+0000 7f2c17400700 1 -- 172.21.15.46:0/1512262263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c08011b50 con 0x7f2c1810fea0 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.457+0000 7f2c1ee00700 1 -- 172.21.15.46:0/1512262263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 msgr2=0x7f2c181102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.457+0000 7f2c1ee00700 1 --2- 172.21.15.46:0/1512262263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181102c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c080098a0 tx=0x7f2c08009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.458+0000 7f2c1ee00700 1 -- 172.21.15.46:0/1512262263 shutdown_connections 2024-04-17T07:47:04.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.458+0000 7f2c1ee00700 1 --2- 172.21.15.46:0/1512262263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181102c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.458+0000 7f2c1ee00700 1 -- 172.21.15.46:0/1512262263 >> 172.21.15.46:0/1512262263 conn(0x7f2c1807d9c0 msgr2=0x7f2c1807fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.458+0000 7f2c1ee00700 1 -- 172.21.15.46:0/1512262263 shutdown_connections 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.458+0000 7f2c1ee00700 1 -- 172.21.15.46:0/1512262263 wait complete. 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.459+0000 7f2c1ee00700 1 Processor -- start 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.459+0000 7f2c1ee00700 1 -- start start 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.459+0000 7f2c1ee00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.459+0000 7f2c1ee00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1810fad0 con 0x7f2c1810fea0 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.460+0000 7f2c17e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.460+0000 7f2c17e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52890/0 (socket says 172.21.15.46:52890) 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.460+0000 7f2c17e00700 1 -- 172.21.15.46:0/2455487020 learned_addr learned my addr 172.21.15.46:0/2455487020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.460+0000 7f2c17e00700 1 -- 172.21.15.46:0/2455487020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181a2e40 con 0x7f2c1810fea0 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.460+0000 7f2c17e00700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181a2900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009fb0 tx=0x7f2c08011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.461+0000 7f2c15600700 1 -- 172.21.15.46:0/2455487020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c080038b0 con 0x7f2c1810fea0 2024-04-17T07:47:04.577 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.461+0000 7f2c15600700 1 -- 172.21.15.46:0/2455487020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c08011e50 con 0x7f2c1810fea0 2024-04-17T07:47:04.578 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.461+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181a30d0 con 0x7f2c1810fea0 2024-04-17T07:47:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.461+0000 7f2c15600700 1 -- 172.21.15.46:0/2455487020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c08019550 con 0x7f2c1810fea0 2024-04-17T07:47:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.461+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a5dc0 con 0x7f2c1810fea0 2024-04-17T07:47:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.462+0000 7f2c15600700 1 -- 172.21.15.46:0/2455487020 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2c0801a430 con 0x7f2c1810fea0 2024-04-17T07:47:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.462+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c180640b0 con 0x7f2c1810fea0 2024-04-17T07:47:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.462+0000 7f2c15600700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f2bfc0395b0 0x7f2bfc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:04.580 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.463+0000 7f2c15600700 1 -- 172.21.15.46:0/2455487020 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c0804d9d0 con 0x7f2c1810fea0 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.463+0000 7f2c0f400700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f2bfc0395b0 0x7f2bfc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.464+0000 7f2c0f400700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f2bfc0395b0 0x7f2bfc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c00006fd0 tx=0x7f2c00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.468+0000 7f2c15600700 1 -- 172.21.15.46:0/2455487020 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2c08020240 con 0x7f2c1810fea0 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.732+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2c1807fd40 con 0x7f2bfc0395b0 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.742+0000 7f2c15600700 1 -- 172.21.15.46:0/2455487020 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2c1807fd40 con 0x7f2bfc0395b0 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f2bfc0395b0 msgr2=0x7f2bfc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f2bfc0395b0 0x7f2bfc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c00006fd0 tx=0x7f2c00006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 msgr2=0x7f2c181a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181a2900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009fb0 tx=0x7f2c08011bf0 comp rx=0 tx=0).stop 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 shutdown_connections 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f2bfc0395b0 0x7f2bfc03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 --2- 172.21.15.46:0/2455487020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c1810fea0 0x7f2c181a2900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:04.581 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 >> 172.21.15.46:0/2455487020 conn(0x7f2c1807d9c0 msgr2=0x7f2c1807f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.745+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 shutdown_connections 2024-04-17T07:47:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:03.746+0000 7f2c1ee00700 1 -- 172.21.15.46:0/2455487020 wait complete. 2024-04-17T07:47:04.582 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-04-17T07:47:05.023 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:04 smithi046 ceph-mon[32038]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:47:05.023 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:04 smithi046 ceph-mon[32038]: Saving service mon spec with placement count:5 2024-04-17T07:47:05.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:04 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:47:06.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:06 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:47:06.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:06 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:47:06.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:06 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:47:06.708 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.766+0000 7fad6b000700 1 Processor -- start 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.766+0000 7fad6b000700 1 -- start start 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.767+0000 7fad6b000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6407c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.767+0000 7fad6b000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad6407c7d0 con 0x7fad6407dd90 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.767+0000 7fad69c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6407c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.767+0000 7fad69c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6407c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52898/0 (socket says 172.21.15.46:52898) 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.767+0000 7fad69c00700 1 -- 172.21.15.46:0/797178087 learned_addr learned my addr 172.21.15.46:0/797178087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.768+0000 7fad69c00700 1 -- 172.21.15.46:0/797178087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6407c950 con 0x7fad6407dd90 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.768+0000 7fad69c00700 1 --2- 172.21.15.46:0/797178087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6407c290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad580098a0 tx=0x7fad58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10a15b536099e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.768+0000 7fad63e00700 1 -- 172.21.15.46:0/797178087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad58004160 con 0x7fad6407dd90 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.768+0000 7fad63e00700 1 -- 172.21.15.46:0/797178087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad58004300 con 0x7fad6407dd90 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.768+0000 7fad63e00700 1 -- 172.21.15.46:0/797178087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad58011b20 con 0x7fad6407dd90 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.769+0000 7fad6b000700 1 -- 172.21.15.46:0/797178087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 msgr2=0x7fad6407c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.769+0000 7fad6b000700 1 --2- 172.21.15.46:0/797178087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6407c290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad580098a0 tx=0x7fad58009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.769+0000 7fad6b000700 1 -- 172.21.15.46:0/797178087 shutdown_connections 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.769+0000 7fad6b000700 1 --2- 172.21.15.46:0/797178087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6407c290 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:06.714 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.769+0000 7fad6b000700 1 -- 172.21.15.46:0/797178087 >> 172.21.15.46:0/797178087 conn(0x7fad64103760 msgr2=0x7fad64105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.769+0000 7fad6b000700 1 -- 172.21.15.46:0/797178087 shutdown_connections 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.770+0000 7fad6b000700 1 -- 172.21.15.46:0/797178087 wait complete. 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.771+0000 7fad6b000700 1 Processor -- start 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.771+0000 7fad6b000700 1 -- start start 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.771+0000 7fad6b000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.771+0000 7fad6b000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410e700 con 0x7fad6407dd90 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.772+0000 7fad69c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.772+0000 7fad69c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6419e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52908/0 (socket says 172.21.15.46:52908) 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.772+0000 7fad69c00700 1 -- 172.21.15.46:0/3786538914 learned_addr learned my addr 172.21.15.46:0/3786538914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.772+0000 7fad69c00700 1 -- 172.21.15.46:0/3786538914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6419e7e0 con 0x7fad6407dd90 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.772+0000 7fad69c00700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6419e2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad580116a0 tx=0x7fad58011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.773+0000 7fad62000700 1 -- 172.21.15.46:0/3786538914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad58003a40 con 0x7fad6407dd90 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.773+0000 7fad62000700 1 -- 172.21.15.46:0/3786538914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad58003be0 con 0x7fad6407dd90 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.773+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6419ea70 con 0x7fad6407dd90 2024-04-17T07:47:06.715 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.773+0000 7fad62000700 1 -- 172.21.15.46:0/3786538914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad5801a750 con 0x7fad6407dd90 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.773+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6419eed0 con 0x7fad6407dd90 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.774+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad640640b0 con 0x7fad6407dd90 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.774+0000 7fad62000700 1 -- 172.21.15.46:0/3786538914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fad58024aa0 con 0x7fad6407dd90 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.775+0000 7fad62000700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fad4c039560 0x7fad4c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.775+0000 7fad62000700 1 -- 172.21.15.46:0/3786538914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad5804ddc0 con 0x7fad6407dd90 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.775+0000 7fad69200700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fad4c039560 0x7fad4c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.776+0000 7fad69200700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fad4c039560 0x7fad4c03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad50006fd0 tx=0x7fad50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:05.781+0000 7fad62000700 1 -- 172.21.15.46:0/3786538914 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fad58011e20 con 0x7fad6407dd90 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.027+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fad64104990 con 0x7fad4c039560 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.037+0000 7fad62000700 1 -- 172.21.15.46:0/3786538914 <== mgr.14120 v2:172.21.15.46:6800/461113542 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fad64104990 con 0x7fad4c039560 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fad4c039560 msgr2=0x7fad4c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fad4c039560 0x7fad4c03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad50006fd0 tx=0x7fad50006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 msgr2=0x7fad6419e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6419e2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad580116a0 tx=0x7fad58011ba0 comp rx=0 tx=0).stop 2024-04-17T07:47:06.716 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 shutdown_connections 2024-04-17T07:47:06.717 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fad4c039560 0x7fad4c03ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:06.717 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 --2- 172.21.15.46:0/3786538914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad6407dd90 0x7fad6419e2a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:06.717 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 >> 172.21.15.46:0/3786538914 conn(0x7fad64103760 msgr2=0x7fad64104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:06.717 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 shutdown_connections 2024-04-17T07:47:06.717 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:06.040+0000 7fad6b000700 1 -- 172.21.15.46:0/3786538914 wait complete. 2024-04-17T07:47:07.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:07 smithi046 ceph-mon[32038]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:07.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:07 smithi046 ceph-mon[32038]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:47:07.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:07 smithi046 ceph-mon[32038]: Saving service mgr spec with placement count:2 2024-04-17T07:47:08.765 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:08 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1788811860' entity='client.admin' 2024-04-17T07:47:08.782 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.296+0000 7f76c7200700 1 Processor -- start 2024-04-17T07:47:08.782 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.296+0000 7f76c7200700 1 -- start start 2024-04-17T07:47:08.782 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.296+0000 7f76c7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c01087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.296+0000 7f76c7200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0108d80 con 0x7f76c0108390 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.297+0000 7f76bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c01087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.298+0000 7f76bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c01087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.298+0000 7f76bfe00700 1 -- 172.21.15.46:0/1262821639 learned_addr learned my addr 172.21.15.46:0/1262821639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.298+0000 7f76bfe00700 1 -- 172.21.15.46:0/1262821639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c0109540 con 0x7f76c0108390 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.299+0000 7f76bfe00700 1 --2- 172.21.15.46:0/1262821639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c01087b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76b00098a0 tx=0x7f76b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d47334685d2979f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.299+0000 7f76bea00700 1 -- 172.21.15.46:0/1262821639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b0004160 con 0x7f76c0108390 2024-04-17T07:47:08.783 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.299+0000 7f76bea00700 1 -- 172.21.15.46:0/1262821639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b0004300 con 0x7f76c0108390 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.299+0000 7f76bea00700 1 -- 172.21.15.46:0/1262821639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b0011b20 con 0x7f76c0108390 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.300+0000 7f76c7200700 1 -- 172.21.15.46:0/1262821639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 msgr2=0x7f76c01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.300+0000 7f76c7200700 1 --2- 172.21.15.46:0/1262821639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c01087b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76b00098a0 tx=0x7f76b0009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.300+0000 7f76c7200700 1 -- 172.21.15.46:0/1262821639 shutdown_connections 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.300+0000 7f76c7200700 1 --2- 172.21.15.46:0/1262821639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c01087b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.300+0000 7f76c7200700 1 -- 172.21.15.46:0/1262821639 >> 172.21.15.46:0/1262821639 conn(0x7f76c01039b0 msgr2=0x7f76c0105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.300+0000 7f76c7200700 1 -- 172.21.15.46:0/1262821639 shutdown_connections 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.300+0000 7f76c7200700 1 -- 172.21.15.46:0/1262821639 wait complete. 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.301+0000 7f76c7200700 1 Processor -- start 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.302+0000 7f76c7200700 1 -- start start 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.302+0000 7f76c7200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.302+0000 7f76c7200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010c8f0 con 0x7f76c0108390 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.302+0000 7f76bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:08.784 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.302+0000 7f76bfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c019c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52930/0 (socket says 172.21.15.46:52930) 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.302+0000 7f76bfe00700 1 -- 172.21.15.46:0/1788811860 learned_addr learned my addr 172.21.15.46:0/1788811860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.303+0000 7f76bfe00700 1 -- 172.21.15.46:0/1788811860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c019c850 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.303+0000 7f76bfe00700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c019c310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000c00 tx=0x7f76b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.303+0000 7f76bcc00700 1 -- 172.21.15.46:0/1788811860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b0003b60 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.303+0000 7f76bcc00700 1 -- 172.21.15.46:0/1788811860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b001a910 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.303+0000 7f76bcc00700 1 -- 172.21.15.46:0/1788811860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b0024930 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.303+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c019cae0 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.304+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c019cf40 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.305+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c00640b0 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.305+0000 7f76bcc00700 1 -- 172.21.15.46:0/1788811860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f76b0022020 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.306+0000 7f76bcc00700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f76a4039600 0x7f76a403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.306+0000 7f76bcc00700 1 -- 172.21.15.46:0/1788811860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f76b004d860 con 0x7f76c0108390 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.306+0000 7f76bf400700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f76a4039600 0x7f76a403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:08.785 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.307+0000 7f76bf400700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f76a4039600 0x7f76a403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f76ac006fd0 tx=0x7f76ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.311+0000 7f76bcc00700 1 -- 172.21.15.46:0/1788811860 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f76b002dde0 con 0x7f76c0108390 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.558+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f76c019f830 con 0x7f76c0108390 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.561+0000 7f76bcc00700 1 -- 172.21.15.46:0/1788811860 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f76b001ab70 con 0x7f76c0108390 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f76a4039600 msgr2=0x7f76a403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f76a4039600 0x7f76a403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f76ac006fd0 tx=0x7f76ac006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 msgr2=0x7f76c019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c019c310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000c00 tx=0x7f76b0011c10 comp rx=0 tx=0).stop 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 shutdown_connections 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f76a4039600 0x7f76a403bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 --2- 172.21.15.46:0/1788811860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0108390 0x7f76c019c310 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 >> 172.21.15.46:0/1788811860 conn(0x7f76c01039b0 msgr2=0x7f76c0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:08.786 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 shutdown_connections 2024-04-17T07:47:08.787 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:07.565+0000 7f76c7200700 1 -- 172.21.15.46:0/1788811860 wait complete. 2024-04-17T07:47:09.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:09 smithi046 ceph-mon[32038]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:09.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:09 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:47:10.336 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.403+0000 7fa14d200700 1 Processor -- start 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.404+0000 7fa14d200700 1 -- start start 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.404+0000 7fa14d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14807c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.404+0000 7fa14d200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa14807c7d0 con 0x7fa14807dde0 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.405+0000 7fa145e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14807c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.405+0000 7fa145e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14807c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42562/0 (socket says 172.21.15.46:42562) 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.405+0000 7fa145e00700 1 -- 172.21.15.46:0/2908345781 learned_addr learned my addr 172.21.15.46:0/2908345781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.405+0000 7fa145e00700 1 -- 172.21.15.46:0/2908345781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14807c950 con 0x7fa14807dde0 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.405+0000 7fa145e00700 1 --2- 172.21.15.46:0/2908345781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14807c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1380098a0 tx=0x7fa138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b5a6d184e2d9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.406+0000 7fa144a00700 1 -- 172.21.15.46:0/2908345781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa138004160 con 0x7fa14807dde0 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.406+0000 7fa144a00700 1 -- 172.21.15.46:0/2908345781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138004300 con 0x7fa14807dde0 2024-04-17T07:47:10.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.406+0000 7fa144a00700 1 -- 172.21.15.46:0/2908345781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa138011b20 con 0x7fa14807dde0 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.407+0000 7fa14d200700 1 -- 172.21.15.46:0/2908345781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 msgr2=0x7fa14807c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.407+0000 7fa14d200700 1 --2- 172.21.15.46:0/2908345781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14807c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1380098a0 tx=0x7fa138009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.407+0000 7fa14d200700 1 -- 172.21.15.46:0/2908345781 shutdown_connections 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.407+0000 7fa14d200700 1 --2- 172.21.15.46:0/2908345781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14807c290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.407+0000 7fa14d200700 1 -- 172.21.15.46:0/2908345781 >> 172.21.15.46:0/2908345781 conn(0x7fa1481039b0 msgr2=0x7fa148105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.407+0000 7fa14d200700 1 -- 172.21.15.46:0/2908345781 shutdown_connections 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.407+0000 7fa14d200700 1 -- 172.21.15.46:0/2908345781 wait complete. 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.408+0000 7fa14d200700 1 Processor -- start 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.408+0000 7fa14d200700 1 -- start start 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.408+0000 7fa14d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.408+0000 7fa14d200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa14810e950 con 0x7fa14807dde0 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.409+0000 7fa145e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:10.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.409+0000 7fa145e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42564/0 (socket says 172.21.15.46:42564) 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.409+0000 7fa145e00700 1 -- 172.21.15.46:0/4047253471 learned_addr learned my addr 172.21.15.46:0/4047253471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.409+0000 7fa145e00700 1 -- 172.21.15.46:0/4047253471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14819e9b0 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.410+0000 7fa145e00700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14819e470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa138004660 tx=0x7fa138011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.410+0000 7fa13ea00700 1 -- 172.21.15.46:0/4047253471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa138003ca0 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.410+0000 7fa13ea00700 1 -- 172.21.15.46:0/4047253471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa13801a910 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.410+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14819ec40 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.410+0000 7fa13ea00700 1 -- 172.21.15.46:0/4047253471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa138024970 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.410+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14819f0a0 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.411+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1480640b0 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.412+0000 7fa13ea00700 1 -- 172.21.15.46:0/4047253471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa138022020 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.412+0000 7fa13ea00700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa12c0397a0 0x7fa12c03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.413+0000 7fa13ea00700 1 -- 172.21.15.46:0/4047253471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa13804da70 con 0x7fa14807dde0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.413+0000 7fa145400700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa12c0397a0 0x7fa12c03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.414+0000 7fa145400700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa12c0397a0 0x7fa12c03bc60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa134006fd0 tx=0x7fa134006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.418+0000 7fa13ea00700 1 -- 172.21.15.46:0/4047253471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa13800b3e0 con 0x7fa14807dde0 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.648+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa1481a1b30 con 0x7fa14807dde0 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.651+0000 7fa13ea00700 1 -- 172.21.15.46:0/4047253471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa13801ab70 con 0x7fa14807dde0 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.654+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa12c0397a0 msgr2=0x7fa12c03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.654+0000 7fa14d200700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa12c0397a0 0x7fa12c03bc60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa134006fd0 tx=0x7fa134006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.654+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 msgr2=0x7fa14819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.654+0000 7fa14d200700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14819e470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa138004660 tx=0x7fa138011ce0 comp rx=0 tx=0).stop 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.654+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 shutdown_connections 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.654+0000 7fa14d200700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa12c0397a0 0x7fa12c03bc60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.654+0000 7fa14d200700 1 --2- 172.21.15.46:0/4047253471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14807dde0 0x7fa14819e470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.655+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 >> 172.21.15.46:0/4047253471 conn(0x7fa1481039b0 msgr2=0x7fa1481043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.655+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 shutdown_connections 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:09.655+0000 7fa14d200700 1 -- 172.21.15.46:0/4047253471 wait complete. 2024-04-17T07:47:10.340 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-04-17T07:47:10.939 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:10 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4047253471' entity='client.admin' 2024-04-17T07:47:11.962 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:11 smithi046 ceph-mon[32038]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:11.962 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:11 smithi046 ceph-mon[32038]: from='mgr.14120 172.21.15.46:0/2854884416' entity='mgr.x' 2024-04-17T07:47:11.962 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:11 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1627171498' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-17T07:47:12.213 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: ignoring --setuser ceph since I am not root 2024-04-17T07:47:12.213 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: ignoring --setgroup ceph since I am not root 2024-04-17T07:47:13.308 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:12.875+0000 7fc2cd05c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:47:13.308 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:12.999+0000 7fc2cd05c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:47:13.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:13 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1627171498' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-17T07:47:13.309 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:13 smithi046 ceph-mon[32038]: mgrmap e9: x(active, since 26s) 2024-04-17T07:47:13.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.939+0000 7ff386200700 1 Processor -- start 2024-04-17T07:47:13.326 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.939+0000 7ff386200700 1 -- start start 2024-04-17T07:47:13.428 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.939+0000 7ff386200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff380107030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.939+0000 7ff386200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff38007bde0 con 0x7ff380106c10 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.940+0000 7ff37f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff380107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.940+0000 7ff37f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff380107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42570/0 (socket says 172.21.15.46:42570) 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.940+0000 7ff37f400700 1 -- 172.21.15.46:0/3720885311 learned_addr learned my addr 172.21.15.46:0/3720885311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.940+0000 7ff37f400700 1 -- 172.21.15.46:0/3720885311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff380107570 con 0x7ff380106c10 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.941+0000 7ff37f400700 1 --2- 172.21.15.46:0/3720885311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff380107030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098a0 tx=0x7ff370009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93ba8e112adc5d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.941+0000 7ff37e000700 1 -- 172.21.15.46:0/3720885311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff370004160 con 0x7ff380106c10 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.941+0000 7ff37e000700 1 -- 172.21.15.46:0/3720885311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff370004300 con 0x7ff380106c10 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.941+0000 7ff37e000700 1 -- 172.21.15.46:0/3720885311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff370011b20 con 0x7ff380106c10 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.942+0000 7ff386200700 1 -- 172.21.15.46:0/3720885311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 msgr2=0x7ff380107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.942+0000 7ff386200700 1 --2- 172.21.15.46:0/3720885311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff380107030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098a0 tx=0x7ff370009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.942+0000 7ff386200700 1 -- 172.21.15.46:0/3720885311 shutdown_connections 2024-04-17T07:47:13.429 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.942+0000 7ff386200700 1 --2- 172.21.15.46:0/3720885311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff380107030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.942+0000 7ff386200700 1 -- 172.21.15.46:0/3720885311 >> 172.21.15.46:0/3720885311 conn(0x7ff380102830 msgr2=0x7ff380104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.942+0000 7ff386200700 1 -- 172.21.15.46:0/3720885311 shutdown_connections 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.942+0000 7ff386200700 1 -- 172.21.15.46:0/3720885311 wait complete. 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.943+0000 7ff386200700 1 Processor -- start 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.944+0000 7ff386200700 1 -- start start 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.944+0000 7ff386200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff38019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.944+0000 7ff386200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff380108640 con 0x7ff380106c10 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.944+0000 7ff37f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff38019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.944+0000 7ff37f400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff38019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42580/0 (socket says 172.21.15.46:42580) 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.944+0000 7ff37f400700 1 -- 172.21.15.46:0/1627171498 learned_addr learned my addr 172.21.15.46:0/1627171498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.945+0000 7ff37f400700 1 -- 172.21.15.46:0/1627171498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38019e950 con 0x7ff380106c10 2024-04-17T07:47:13.430 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.945+0000 7ff37f400700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff38019e410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3700117e0 tx=0x7ff370011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.945+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff370003ba0 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.945+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37001a910 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.945+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3700248f0 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.945+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38019ebe0 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.946+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38019f040 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.947+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3800640b0 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.947+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff370022020 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.948+0000 7ff377e00700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7ff364039600 0x7ff36403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.948+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff37004dae0 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.948+0000 7ff37ea00700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7ff364039600 0x7ff36403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.949+0000 7ff37ea00700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7ff364039600 0x7ff36403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006fd0 tx=0x7ff36c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:10.953+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff37001ab70 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:11.254+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff380063660 con 0x7ff380106c10 2024-04-17T07:47:13.431 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.027+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff370017ca0 con 0x7ff380106c10 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.032+0000 7ff377e00700 1 -- 172.21.15.46:0/1627171498 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff37001a430 con 0x7ff380106c10 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7ff364039600 msgr2=0x7ff36403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7ff364039600 0x7ff36403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff36c006fd0 tx=0x7ff36c006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 msgr2=0x7ff38019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff38019e410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3700117e0 tx=0x7ff370011ba0 comp rx=0 tx=0).stop 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 shutdown_connections 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7ff364039600 0x7ff36403bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 --2- 172.21.15.46:0/1627171498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff380106c10 0x7ff38019e410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 >> 172.21.15.46:0/1627171498 conn(0x7ff380102830 msgr2=0x7ff380195020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 shutdown_connections 2024-04-17T07:47:13.432 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:12.033+0000 7ff386200700 1 -- 172.21.15.46:0/1627171498 wait complete. 2024-04-17T07:47:13.964 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:13.868+0000 7fc2cd05c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:47:14.246 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:14.003+0000 7fc2cd05c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:47:14.246 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:14.189+0000 7fc2cd05c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:47:14.544 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:14.321+0000 7fc2cd05c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:47:14.544 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:14.437+0000 7fc2cd05c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:47:14.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:14 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2361362437' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-17T07:47:14.860 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:14.786+0000 7fc2cd05c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:47:15.185 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:14.936+0000 7fc2cd05c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:47:15.185 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:15 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:15.055+0000 7fc2cd05c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:47:15.292 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:47:15.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-17T07:47:15.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T07:47:15.293 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-17T07:47:15.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-17T07:47:15.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:47:15.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.244+0000 7f59e1200700 1 Processor -- start 2024-04-17T07:47:15.294 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.245+0000 7f59e1200700 1 -- start start 2024-04-17T07:47:15.295 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.245+0000 7f59e1200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.245+0000 7f59e1200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc07c8a0 con 0x7f59dc07de20 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.246+0000 7f59d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.246+0000 7f59d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42594/0 (socket says 172.21.15.46:42594) 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.246+0000 7f59d9e00700 1 -- 172.21.15.46:0/2240673691 learned_addr learned my addr 172.21.15.46:0/2240673691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.246+0000 7f59d9e00700 1 -- 172.21.15.46:0/2240673691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc07ca20 con 0x7f59dc07de20 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.247+0000 7f59d9e00700 1 --2- 172.21.15.46:0/2240673691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc07c2d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098a0 tx=0x7f59cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d169931f916876 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.247+0000 7f59d8a00700 1 -- 172.21.15.46:0/2240673691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59cc004160 con 0x7f59dc07de20 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.247+0000 7f59d8a00700 1 -- 172.21.15.46:0/2240673691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59cc004300 con 0x7f59dc07de20 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.247+0000 7f59d8a00700 1 -- 172.21.15.46:0/2240673691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59cc011b20 con 0x7f59dc07de20 2024-04-17T07:47:15.296 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.248+0000 7f59e1200700 1 -- 172.21.15.46:0/2240673691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 msgr2=0x7f59dc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.248+0000 7f59e1200700 1 --2- 172.21.15.46:0/2240673691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc07c2d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098a0 tx=0x7f59cc009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.248+0000 7f59e1200700 1 -- 172.21.15.46:0/2240673691 shutdown_connections 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.248+0000 7f59e1200700 1 --2- 172.21.15.46:0/2240673691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc07c2d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.248+0000 7f59e1200700 1 -- 172.21.15.46:0/2240673691 >> 172.21.15.46:0/2240673691 conn(0x7f59dc1039b0 msgr2=0x7f59dc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.248+0000 7f59e1200700 1 -- 172.21.15.46:0/2240673691 shutdown_connections 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.248+0000 7f59e1200700 1 -- 172.21.15.46:0/2240673691 wait complete. 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.249+0000 7f59e1200700 1 Processor -- start 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.249+0000 7f59e1200700 1 -- start start 2024-04-17T07:47:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.250+0000 7f59e1200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.250+0000 7f59e1200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10e970 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.250+0000 7f59d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.250+0000 7f59d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42604/0 (socket says 172.21.15.46:42604) 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.250+0000 7f59d9e00700 1 -- 172.21.15.46:0/2361362437 learned_addr learned my addr 172.21.15.46:0/2361362437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.250+0000 7f59d9e00700 1 -- 172.21.15.46:0/2361362437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc19ea40 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.251+0000 7f59d9e00700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc19e500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59cc011b60 tx=0x7f59cc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.251+0000 7f59d2a00700 1 -- 172.21.15.46:0/2361362437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59cc003b60 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.251+0000 7f59d2a00700 1 -- 172.21.15.46:0/2361362437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59cc01a910 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.251+0000 7f59d2a00700 1 -- 172.21.15.46:0/2361362437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59cc024930 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.251+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc19ecd0 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.251+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc19f130 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.252+0000 7f59d2a00700 1 -- 172.21.15.46:0/2361362437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f59cc022020 con 0x7f59dc07de20 2024-04-17T07:47:15.303 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.252+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc0640b0 con 0x7f59dc07de20 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.253+0000 7f59d2a00700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 0x7f59c003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.253+0000 7f59d2a00700 1 -- 172.21.15.46:0/2361362437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59cc04d940 con 0x7f59dc07de20 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.253+0000 7f59d9400700 1 -- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 msgr2=0x7f59c003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/461113542 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.253+0000 7f59d9400700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 0x7f59c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.258+0000 7f59d2a00700 1 -- 172.21.15.46:0/2361362437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f59cc00b3e0 con 0x7f59dc07de20 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.455+0000 7f59d9400700 1 -- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 msgr2=0x7f59c003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/461113542 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.455+0000 7f59d9400700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 0x7f59c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.544+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f59dc1a1ba0 con 0x7f59dc07de20 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.545+0000 7f59d2a00700 1 -- 172.21.15.46:0/2361362437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f59cc04d290 con 0x7f59dc07de20 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.548+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 msgr2=0x7f59c003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.548+0000 7f59e1200700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 0x7f59c003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.548+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 msgr2=0x7f59dc19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.548+0000 7f59e1200700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc19e500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59cc011b60 tx=0x7f59cc011c40 comp rx=0 tx=0).stop 2024-04-17T07:47:15.304 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.549+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 shutdown_connections 2024-04-17T07:47:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.549+0000 7f59e1200700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7f59c0039660 0x7f59c003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.549+0000 7f59e1200700 1 --2- 172.21.15.46:0/2361362437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59dc07de20 0x7f59dc19e500 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.549+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 >> 172.21.15.46:0/2361362437 conn(0x7f59dc1039b0 msgr2=0x7f59dc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.549+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 shutdown_connections 2024-04-17T07:47:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:14.549+0000 7f59e1200700 1 -- 172.21.15.46:0/2361362437 wait complete. 2024-04-17T07:47:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-04-17T07:47:15.305 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-04-17T07:47:15.455 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:15 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:15.185+0000 7fc2cd05c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:47:15.455 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:15 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:15.421+0000 7fc2cd05c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:47:17.110 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:16.789+0000 7fc2cd05c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:47:17.110 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:16.968+0000 7fc2cd05c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:47:17.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:17 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:17.401+0000 7fc2cd05c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:47:17.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:17 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:17.526+0000 7fc2cd05c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:47:18.056 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:17 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:17.799+0000 7fc2cd05c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:47:18.056 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:18.055+0000 7fc2cd05c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:47:18.780 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:18.453+0000 7fc2cd05c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:47:18.781 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:18.780+0000 7fc2cd05c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:47:19.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:19 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:19.101+0000 7fc2cd05c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:47:20.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:20 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:20.185+0000 7fc2cd05c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:47:21.348 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:20 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:20.912+0000 7fc2cd05c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:47:21.348 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:21.047+0000 7fc2cd05c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:47:21.610 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:47:21.347+0000 7fc2cd05c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:47:21.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: Active manager daemon x restarted 2024-04-17T07:47:21.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: Activating manager daemon x 2024-04-17T07:47:21.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: osdmap e3: 0 total, 0 up, 0 in 2024-04-17T07:47:21.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: mgrmap e10: x(active, starting, since 0.00840996s) 2024-04-17T07:47:21.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:47:21.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T07:47:21.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T07:47:21.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T07:47:21.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T07:47:21.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:21 smithi046 ceph-mon[32038]: Manager daemon x is now available 2024-04-17T07:47:22.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:47:22.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T07:47:22.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T07:47:22.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:22 smithi046 ceph-mon[32038]: mgrmap e11: x(active, since 1.01141s) 2024-04-17T07:47:23.137 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:47:23.137 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-17T07:47:23.137 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.883+0000 7fa932c00700 1 Processor -- start 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.884+0000 7fa932c00700 1 -- start start 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.884+0000 7fa932c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c10a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.884+0000 7fa932c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c10af50 con 0x7fa92c10a560 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.885+0000 7fa92be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c10a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.885+0000 7fa92be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c10a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42610/0 (socket says 172.21.15.46:42610) 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.885+0000 7fa92be00700 1 -- 172.21.15.46:0/2032089079 learned_addr learned my addr 172.21.15.46:0/2032089079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.885+0000 7fa92be00700 1 -- 172.21.15.46:0/2032089079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c10b710 con 0x7fa92c10a560 2024-04-17T07:47:23.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.885+0000 7fa92be00700 1 --2- 172.21.15.46:0/2032089079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c10a980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0098a0 tx=0x7fa91c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568f1b83c073df8b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.886+0000 7fa92b400700 1 -- 172.21.15.46:0/2032089079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c004160 con 0x7fa92c10a560 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.886+0000 7fa92b400700 1 -- 172.21.15.46:0/2032089079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa91c004300 con 0x7fa92c10a560 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.886+0000 7fa92b400700 1 -- 172.21.15.46:0/2032089079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c011b20 con 0x7fa92c10a560 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.886+0000 7fa932c00700 1 -- 172.21.15.46:0/2032089079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 msgr2=0x7fa92c10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.886+0000 7fa932c00700 1 --2- 172.21.15.46:0/2032089079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c10a980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0098a0 tx=0x7fa91c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.887+0000 7fa932c00700 1 -- 172.21.15.46:0/2032089079 shutdown_connections 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.887+0000 7fa932c00700 1 --2- 172.21.15.46:0/2032089079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c10a980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.887+0000 7fa932c00700 1 -- 172.21.15.46:0/2032089079 >> 172.21.15.46:0/2032089079 conn(0x7fa92c07d990 msgr2=0x7fa92c108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.887+0000 7fa932c00700 1 -- 172.21.15.46:0/2032089079 shutdown_connections 2024-04-17T07:47:23.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.887+0000 7fa932c00700 1 -- 172.21.15.46:0/2032089079 wait complete. 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.888+0000 7fa932c00700 1 Processor -- start 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.888+0000 7fa932c00700 1 -- start start 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.888+0000 7fa932c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.888+0000 7fa932c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c10eac0 con 0x7fa92c10a560 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.888+0000 7fa92be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.888+0000 7fa92be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42616/0 (socket says 172.21.15.46:42616) 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.888+0000 7fa92be00700 1 -- 172.21.15.46:0/3673807883 learned_addr learned my addr 172.21.15.46:0/3673807883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.889+0000 7fa92be00700 1 -- 172.21.15.46:0/3673807883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c19e930 con 0x7fa92c10a560 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.889+0000 7fa92be00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c19e3f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0117e0 tx=0x7fa91c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.889+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c003bf0 con 0x7fa92c10a560 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.889+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa91c01a910 con 0x7fa92c10a560 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.889+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c19ebc0 con 0x7fa92c10a560 2024-04-17T07:47:23.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.890+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c024970 con 0x7fa92c10a560 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.890+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c19f020 con 0x7fa92c10a560 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.891+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa91c022020 con 0x7fa92c10a560 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.892+0000 7fa929600700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.892+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 --> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa91003c1d0 con 0x7fa910039600 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.892+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa91c04dee0 con 0x7fa92c10a560 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.892+0000 7fa923e00700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 msgr2=0x7fa91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/461113542 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:15.892+0000 7fa923e00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:16.093+0000 7fa923e00700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 msgr2=0x7fa91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/461113542 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:16.093+0000 7fa923e00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:16.494+0000 7fa923e00700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 msgr2=0x7fa91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/461113542 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:16.494+0000 7fa923e00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:17.295+0000 7fa923e00700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 msgr2=0x7fa91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/461113542 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:17.295+0000 7fa923e00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:18.897+0000 7fa923e00700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 msgr2=0x7fa91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/461113542 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:18.897+0000 7fa923e00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-17T07:47:23.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:21.356+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fa91c02d3f0 con 0x7fa92c10a560 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:21.356+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 msgr2=0x7fa91003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:21.356+0000 7fa929600700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.358+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fa91c04fe70 con 0x7fa92c10a560 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.358+0000 7fa929600700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa91003cfe0 0x7fa91003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.359+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa91c01eec0 con 0x7fa91003cfe0 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.360+0000 7fa923e00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa91003cfe0 0x7fa91003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.361+0000 7fa923e00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa91003cfe0 0x7fa91003f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa914003a10 tx=0x7fa914009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.362+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa91c01eec0 con 0x7fa91003cfe0 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.370+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa92c0640b0 con 0x7fa91003cfe0 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa929600700 1 -- 172.21.15.46:0/3673807883 <== mgr.14152 v2:172.21.15.46:6800/4172081855 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa92c0640b0 con 0x7fa91003cfe0 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa91003cfe0 msgr2=0x7fa91003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa91003cfe0 0x7fa91003f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa914003a10 tx=0x7fa914009270 comp rx=0 tx=0).stop 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 msgr2=0x7fa92c19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:23.142 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c19e3f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0117e0 tx=0x7fa91c011ba0 comp rx=0 tx=0).stop 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 shutdown_connections 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa91003cfe0 0x7fa91003f3d0 secure :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa914003a10 tx=0x7fa914009270 comp rx=0 tx=0).stop 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa92c10a560 0x7fa92c19e3f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 --2- 172.21.15.46:0/3673807883 >> [v2:172.21.15.46:6800/461113542,v1:172.21.15.46:6801/461113542] conn(0x7fa910039600 0x7fa91003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.371+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 >> 172.21.15.46:0/3673807883 conn(0x7fa92c07d990 msgr2=0x7fa92c107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.372+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 shutdown_connections 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:22.372+0000 7fa932c00700 1 -- 172.21.15.46:0/3673807883 wait complete. 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-04-17T07:47:23.143 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-04-17T07:47:23.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:23 smithi046 ceph-mon[32038]: [17/Apr/2024:07:47:22] ENGINE Bus STARTING 2024-04-17T07:47:23.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:23 smithi046 ceph-mon[32038]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-17T07:47:23.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:23 smithi046 ceph-mon[32038]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-17T07:47:23.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:23 smithi046 ceph-mon[32038]: [17/Apr/2024:07:47:22] ENGINE Serving on https://172.21.15.46:7150 2024-04-17T07:47:23.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:23 smithi046 ceph-mon[32038]: [17/Apr/2024:07:47:22] ENGINE Serving on http://172.21.15.46:8765 2024-04-17T07:47:23.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:23 smithi046 ceph-mon[32038]: [17/Apr/2024:07:47:22] ENGINE Bus STARTED 2024-04-17T07:47:24.587 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:24 smithi046 ceph-mon[32038]: mgrmap e12: x(active, since 2s) 2024-04-17T07:47:24.587 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:24 smithi046 ceph-mon[32038]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:47:24.588 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:24.588 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.847+0000 7f220a600700 1 Processor -- start 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.847+0000 7f220a600700 1 -- start start 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.848+0000 7f220a600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220407c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.848+0000 7f220a600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f220407c840 con 0x7f220407de10 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.848+0000 7f2209200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.848+0000 7f2209200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220407c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43140/0 (socket says 172.21.15.46:43140) 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.848+0000 7f2209200700 1 -- 172.21.15.46:0/2546272982 learned_addr learned my addr 172.21.15.46:0/2546272982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.849+0000 7f2209200700 1 -- 172.21.15.46:0/2546272982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220407c9c0 con 0x7f220407de10 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.849+0000 7f2209200700 1 --2- 172.21.15.46:0/2546272982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220407c270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21f80098a0 tx=0x7f21f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f46193ba95baaa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.849+0000 7f2203400700 1 -- 172.21.15.46:0/2546272982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f8004160 con 0x7f220407de10 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.849+0000 7f2203400700 1 -- 172.21.15.46:0/2546272982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f8004300 con 0x7f220407de10 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.849+0000 7f2203400700 1 -- 172.21.15.46:0/2546272982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f8011b20 con 0x7f220407de10 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.850+0000 7f220a600700 1 -- 172.21.15.46:0/2546272982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 msgr2=0x7f220407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.850+0000 7f220a600700 1 --2- 172.21.15.46:0/2546272982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220407c270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21f80098a0 tx=0x7f21f8009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.850+0000 7f220a600700 1 -- 172.21.15.46:0/2546272982 shutdown_connections 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.850+0000 7f220a600700 1 --2- 172.21.15.46:0/2546272982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220407c270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.850+0000 7f220a600700 1 -- 172.21.15.46:0/2546272982 >> 172.21.15.46:0/2546272982 conn(0x7f2204103720 msgr2=0x7f2204105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:24.937 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.850+0000 7f220a600700 1 -- 172.21.15.46:0/2546272982 shutdown_connections 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.850+0000 7f220a600700 1 -- 172.21.15.46:0/2546272982 wait complete. 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.851+0000 7f220a600700 1 Processor -- start 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.851+0000 7f220a600700 1 -- start start 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.852+0000 7f220a600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.852+0000 7f220a600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f220410e780 con 0x7f220407de10 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.852+0000 7f2209200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.852+0000 7f2209200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220419e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43142/0 (socket says 172.21.15.46:43142) 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.852+0000 7f2209200700 1 -- 172.21.15.46:0/820646988 learned_addr learned my addr 172.21.15.46:0/820646988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.852+0000 7f2209200700 1 -- 172.21.15.46:0/820646988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220419e840 con 0x7f220407de10 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.853+0000 7f2209200700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220419e300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21f8011b60 tx=0x7f21f8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.853+0000 7f2201600700 1 -- 172.21.15.46:0/820646988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f8003b60 con 0x7f220407de10 2024-04-17T07:47:24.938 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.853+0000 7f2201600700 1 -- 172.21.15.46:0/820646988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f801a910 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.853+0000 7f2201600700 1 -- 172.21.15.46:0/820646988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f8024930 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.853+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220419ead0 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.853+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220419ef30 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.854+0000 7f2201600700 1 -- 172.21.15.46:0/820646988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f21f8022020 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.854+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22040640b0 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.855+0000 7f2201600700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f21ec039660 0x7f21ec03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.855+0000 7f2201600700 1 -- 172.21.15.46:0/820646988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f21f804daa0 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.855+0000 7f2203e00700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f21ec039660 0x7f21ec03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.856+0000 7f2203e00700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f21ec039660 0x7f21ec03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21f0006fd0 tx=0x7f21f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:23.860+0000 7f2201600700 1 -- 172.21.15.46:0/820646988 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f21f800b3e0 con 0x7f220407de10 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.096+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f2204104960 con 0x7f21ec039660 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.178+0000 7f2201600700 1 -- 172.21.15.46:0/820646988 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f2204104960 con 0x7f21ec039660 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f21ec039660 msgr2=0x7f21ec03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f21ec039660 0x7f21ec03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21f0006fd0 tx=0x7f21f0006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 msgr2=0x7f220419e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:24.939 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220419e300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21f8011b60 tx=0x7f21f8011c40 comp rx=0 tx=0).stop 2024-04-17T07:47:24.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 shutdown_connections 2024-04-17T07:47:24.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f21ec039660 0x7f21ec03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:24.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 --2- 172.21.15.46:0/820646988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f220407de10 0x7f220419e300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:24.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 >> 172.21.15.46:0/820646988 conn(0x7f2204103720 msgr2=0x7f2204104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:24.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 shutdown_connections 2024-04-17T07:47:24.940 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:24.181+0000 7f220a600700 1 -- 172.21.15.46:0/820646988 wait complete. 2024-04-17T07:47:24.940 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-04-17T07:47:25.816 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:26.801 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$/CmD6nT7xYB2Bx0jBzPlruaXah/TZsNxvqaagoq3stNgfXvJttEMy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713340046, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-17T07:47:26.801 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.547+0000 7f683c600700 1 Processor -- start 2024-04-17T07:47:26.801 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.547+0000 7f683c600700 1 -- start start 2024-04-17T07:47:26.801 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.548+0000 7f683c600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683410b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.548+0000 7f683c600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f683407be60 con 0x7f6834108da0 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.548+0000 7f6839c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683410b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.548+0000 7f6839c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683410b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43152/0 (socket says 172.21.15.46:43152) 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.548+0000 7f6839c00700 1 -- 172.21.15.46:0/908406652 learned_addr learned my addr 172.21.15.46:0/908406652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.549+0000 7f6839c00700 1 -- 172.21.15.46:0/908406652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683410b6d0 con 0x7f6834108da0 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.549+0000 7f6839c00700 1 --2- 172.21.15.46:0/908406652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683410b190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f68280098a0 tx=0x7f6828009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42f928fcdc28195 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.549+0000 7f6833e00700 1 -- 172.21.15.46:0/908406652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6828004160 con 0x7f6834108da0 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.549+0000 7f6833e00700 1 -- 172.21.15.46:0/908406652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6828004300 con 0x7f6834108da0 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.549+0000 7f6833e00700 1 -- 172.21.15.46:0/908406652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6828011b20 con 0x7f6834108da0 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.550+0000 7f683c600700 1 -- 172.21.15.46:0/908406652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 msgr2=0x7f683410b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.550+0000 7f683c600700 1 --2- 172.21.15.46:0/908406652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683410b190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f68280098a0 tx=0x7f6828009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.550+0000 7f683c600700 1 -- 172.21.15.46:0/908406652 shutdown_connections 2024-04-17T07:47:26.802 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.550+0000 7f683c600700 1 --2- 172.21.15.46:0/908406652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683410b190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.550+0000 7f683c600700 1 -- 172.21.15.46:0/908406652 >> 172.21.15.46:0/908406652 conn(0x7f68341027f0 msgr2=0x7f6834104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.550+0000 7f683c600700 1 -- 172.21.15.46:0/908406652 shutdown_connections 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.550+0000 7f683c600700 1 -- 172.21.15.46:0/908406652 wait complete. 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.551+0000 7f683c600700 1 Processor -- start 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.552+0000 7f683c600700 1 -- start start 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.552+0000 7f683c600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683419e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.552+0000 7f683c600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f683410c740 con 0x7f6834108da0 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.552+0000 7f6839c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683419e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.552+0000 7f6839c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683419e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43156/0 (socket says 172.21.15.46:43156) 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.552+0000 7f6839c00700 1 -- 172.21.15.46:0/1701956984 learned_addr learned my addr 172.21.15.46:0/1701956984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.553+0000 7f6839c00700 1 -- 172.21.15.46:0/1701956984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683419ea00 con 0x7f6834108da0 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.553+0000 7f6839c00700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683419e4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6828011b60 tx=0x7f6828011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.553+0000 7f6832000700 1 -- 172.21.15.46:0/1701956984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6828003b60 con 0x7f6834108da0 2024-04-17T07:47:26.803 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.553+0000 7f6832000700 1 -- 172.21.15.46:0/1701956984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682801a910 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.553+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683419ec90 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.553+0000 7f6832000700 1 -- 172.21.15.46:0/1701956984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6828024930 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.553+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683419f0f0 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.555+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68340640b0 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.555+0000 7f6832000700 1 -- 172.21.15.46:0/1701956984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6828022020 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.555+0000 7f6832000700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f681c039660 0x7f681c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.555+0000 7f6832000700 1 -- 172.21.15.46:0/1701956984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f682804da70 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.556+0000 7f6839200700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f681c039660 0x7f681c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.556+0000 7f6839200700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f681c039660 0x7f681c03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6820006fd0 tx=0x7f6820006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.561+0000 7f6832000700 1 -- 172.21.15.46:0/1701956984 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f682800edf0 con 0x7f6834108da0 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:25.815+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f68341a1b00 con 0x7f681c039660 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.070+0000 7f6832000700 1 -- 172.21.15.46:0/1701956984 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f68341a1b00 con 0x7f681c039660 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f681c039660 msgr2=0x7f681c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f681c039660 0x7f681c03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6820006fd0 tx=0x7f6820006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 msgr2=0x7f683419e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683419e4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6828011b60 tx=0x7f6828011c40 comp rx=0 tx=0).stop 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 shutdown_connections 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f681c039660 0x7f681c03bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 --2- 172.21.15.46:0/1701956984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6834108da0 0x7f683419e4c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 >> 172.21.15.46:0/1701956984 conn(0x7f68341027f0 msgr2=0x7f6834103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.073+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 shutdown_connections 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:26.074+0000 7f683c600700 1 -- 172.21.15.46:0/1701956984 wait complete. 2024-04-17T07:47:26.805 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-04-17T07:47:27.349 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:27 smithi046 ceph-mon[32038]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:47:27.349 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:27.349 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:28.325 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:28 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1453544148' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.446+0000 7f8e07200700 1 Processor -- start 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.446+0000 7f8e07200700 1 -- start start 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.446+0000 7f8e07200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e08106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.446+0000 7f8e07200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0807be20 con 0x7f8e081069b0 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.447+0000 7f8e05e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e08106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.447+0000 7f8e05e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e08106dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43158/0 (socket says 172.21.15.46:43158) 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.447+0000 7f8e05e00700 1 -- 172.21.15.46:0/389571306 learned_addr learned my addr 172.21.15.46:0/389571306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.448+0000 7f8e05e00700 1 -- 172.21.15.46:0/389571306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e08107310 con 0x7f8e081069b0 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.448+0000 7f8e05e00700 1 --2- 172.21.15.46:0/389571306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e08106dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8df80098a0 tx=0x7f8df8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6e478a1a7da2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:29.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.448+0000 7f8e04a00700 1 -- 172.21.15.46:0/389571306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df8004160 con 0x7f8e081069b0 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.448+0000 7f8e04a00700 1 -- 172.21.15.46:0/389571306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df8004300 con 0x7f8e081069b0 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.448+0000 7f8e04a00700 1 -- 172.21.15.46:0/389571306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df8011b20 con 0x7f8e081069b0 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.449+0000 7f8e07200700 1 -- 172.21.15.46:0/389571306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 msgr2=0x7f8e08106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.449+0000 7f8e07200700 1 --2- 172.21.15.46:0/389571306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e08106dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8df80098a0 tx=0x7f8df8009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.449+0000 7f8e07200700 1 -- 172.21.15.46:0/389571306 shutdown_connections 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.449+0000 7f8e07200700 1 --2- 172.21.15.46:0/389571306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e08106dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.449+0000 7f8e07200700 1 -- 172.21.15.46:0/389571306 >> 172.21.15.46:0/389571306 conn(0x7f8e08102610 msgr2=0x7f8e08104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.449+0000 7f8e07200700 1 -- 172.21.15.46:0/389571306 shutdown_connections 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.449+0000 7f8e07200700 1 -- 172.21.15.46:0/389571306 wait complete. 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.450+0000 7f8e07200700 1 Processor -- start 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.451+0000 7f8e07200700 1 -- start start 2024-04-17T07:47:29.022 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.451+0000 7f8e07200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e0819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:29.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.451+0000 7f8e07200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081083e0 con 0x7f8e081069b0 2024-04-17T07:47:29.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.451+0000 7f8e05e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e0819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:29.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.451+0000 7f8e05e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e0819e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43174/0 (socket says 172.21.15.46:43174) 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.451+0000 7f8e05e00700 1 -- 172.21.15.46:0/1453544148 learned_addr learned my addr 172.21.15.46:0/1453544148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.452+0000 7f8e05e00700 1 -- 172.21.15.46:0/1453544148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0819e790 con 0x7f8e081069b0 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.452+0000 7f8e05e00700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e0819e250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8df80117e0 tx=0x7f8df8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.452+0000 7f8dfea00700 1 -- 172.21.15.46:0/1453544148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df8003bf0 con 0x7f8e081069b0 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.452+0000 7f8dfea00700 1 -- 172.21.15.46:0/1453544148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df801a910 con 0x7f8e081069b0 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.453+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0819ea20 con 0x7f8e081069b0 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.453+0000 7f8dfea00700 1 -- 172.21.15.46:0/1453544148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df8024970 con 0x7f8e081069b0 2024-04-17T07:47:29.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.453+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0819ee80 con 0x7f8e081069b0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.454+0000 7f8dfea00700 1 -- 172.21.15.46:0/1453544148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f8df8022020 con 0x7f8e081069b0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.454+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e080640b0 con 0x7f8e081069b0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.454+0000 7f8dfea00700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8dec039610 0x7f8dec03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.454+0000 7f8dfea00700 1 -- 172.21.15.46:0/1453544148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8df804dc60 con 0x7f8e081069b0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.455+0000 7f8e05400700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8dec039610 0x7f8dec03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.455+0000 7f8e05400700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8dec039610 0x7f8dec03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006fd0 tx=0x7f8df4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.460+0000 7f8dfea00700 1 -- 172.21.15.46:0/1453544148 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8df8011e20 con 0x7f8e081069b0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.693+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f8e08063660 con 0x7f8e081069b0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.694+0000 7f8dfea00700 1 -- 172.21.15.46:0/1453544148 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f8df800ba30 con 0x7f8e081069b0 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8dec039610 msgr2=0x7f8dec03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8dec039610 0x7f8dec03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006fd0 tx=0x7f8df4006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 msgr2=0x7f8e0819e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e0819e250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8df80117e0 tx=0x7f8df8011c10 comp rx=0 tx=0).stop 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 shutdown_connections 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8dec039610 0x7f8dec03bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 --2- 172.21.15.46:0/1453544148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e081069b0 0x7f8e0819e250 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 >> 172.21.15.46:0/1453544148 conn(0x7f8e08102610 msgr2=0x7f8e08194f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 shutdown_connections 2024-04-17T07:47:29.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:27.697+0000 7f8e07200700 1 -- 172.21.15.46:0/1453544148 wait complete. 2024-04-17T07:47:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:47:29.054 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-04-17T07:47:29.081 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:47:29.081 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-04-17T07:47:29.081 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-04-17T07:47:29.081 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-17T07:47:29.089 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-04-17T07:47:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-04-17T07:47:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-04-17T07:47:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: Password: m1y00nxewa 2024-04-17T07:47:29.089 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:29.091 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config directory 2024-04-17T07:47:29.093 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling autotune for osd_memory_target 2024-04-17T07:47:29.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:29 smithi046 ceph-mon[32038]: mgrmap e13: x(active, since 6s) 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.029+0000 7fa6e1a00700 1 Processor -- start 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.029+0000 7fa6e1a00700 1 -- start start 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.029+0000 7fa6e1a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc07c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.030+0000 7fa6e1a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc07c7d0 con 0x7fa6dc07dde0 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.030+0000 7fa6daa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc07c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.030+0000 7fa6daa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc07c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48068/0 (socket says 172.21.15.46:48068) 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.030+0000 7fa6daa00700 1 -- 172.21.15.46:0/2775676288 learned_addr learned my addr 172.21.15.46:0/2775676288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.031+0000 7fa6daa00700 1 -- 172.21.15.46:0/2775676288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc07c950 con 0x7fa6dc07dde0 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.031+0000 7fa6daa00700 1 --2- 172.21.15.46:0/2775676288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc07c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0098a0 tx=0x7fa6cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2067bc6c73685fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.031+0000 7fa6da000700 1 -- 172.21.15.46:0/2775676288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6cc004160 con 0x7fa6dc07dde0 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.031+0000 7fa6da000700 1 -- 172.21.15.46:0/2775676288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc004300 con 0x7fa6dc07dde0 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.031+0000 7fa6da000700 1 -- 172.21.15.46:0/2775676288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6cc011b20 con 0x7fa6dc07dde0 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.032+0000 7fa6e1a00700 1 -- 172.21.15.46:0/2775676288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 msgr2=0x7fa6dc07c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.032+0000 7fa6e1a00700 1 --2- 172.21.15.46:0/2775676288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc07c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0098a0 tx=0x7fa6cc009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.032+0000 7fa6e1a00700 1 -- 172.21.15.46:0/2775676288 shutdown_connections 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.032+0000 7fa6e1a00700 1 --2- 172.21.15.46:0/2775676288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc07c290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.032+0000 7fa6e1a00700 1 -- 172.21.15.46:0/2775676288 >> 172.21.15.46:0/2775676288 conn(0x7fa6dc1039b0 msgr2=0x7fa6dc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.033+0000 7fa6e1a00700 1 -- 172.21.15.46:0/2775676288 shutdown_connections 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.033+0000 7fa6e1a00700 1 -- 172.21.15.46:0/2775676288 wait complete. 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.034+0000 7fa6e1a00700 1 Processor -- start 2024-04-17T07:47:30.910 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.034+0000 7fa6e1a00700 1 -- start start 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.034+0000 7fa6e1a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.034+0000 7fa6e1a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc10e950 con 0x7fa6dc07dde0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.034+0000 7fa6daa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.035+0000 7fa6daa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48078/0 (socket says 172.21.15.46:48078) 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.035+0000 7fa6daa00700 1 -- 172.21.15.46:0/745822404 learned_addr learned my addr 172.21.15.46:0/745822404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.035+0000 7fa6daa00700 1 -- 172.21.15.46:0/745822404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc19e9b0 con 0x7fa6dc07dde0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.035+0000 7fa6daa00700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc19e470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0116a0 tx=0x7fa6cc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.035+0000 7fa6d3e00700 1 -- 172.21.15.46:0/745822404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6cc003ac0 con 0x7fa6dc07dde0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.036+0000 7fa6d3e00700 1 -- 172.21.15.46:0/745822404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc003c60 con 0x7fa6dc07dde0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.036+0000 7fa6d3e00700 1 -- 172.21.15.46:0/745822404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6cc01a7d0 con 0x7fa6dc07dde0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.036+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc19ec40 con 0x7fa6dc07dde0 2024-04-17T07:47:30.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.036+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc19f0a0 con 0x7fa6dc07dde0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.037+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc0640b0 con 0x7fa6dc07dde0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.037+0000 7fa6d3e00700 1 -- 172.21.15.46:0/745822404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa6cc024b30 con 0x7fa6dc07dde0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.037+0000 7fa6d3e00700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6c00396b0 0x7fa6c003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.037+0000 7fa6d3e00700 1 -- 172.21.15.46:0/745822404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa6cc04de50 con 0x7fa6dc07dde0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.038+0000 7fa6d2000700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6c00396b0 0x7fa6c003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.038+0000 7fa6d2000700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6c00396b0 0x7fa6c003bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4006fd0 tx=0x7fa6c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.043+0000 7fa6d3e00700 1 -- 172.21.15.46:0/745822404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa6cc011e20 con 0x7fa6dc07dde0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.271+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa6dc1a1b30 con 0x7fa6dc07dde0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.272+0000 7fa6d3e00700 1 -- 172.21.15.46:0/745822404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa6cc04f0c0 con 0x7fa6dc07dde0 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.275+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6c00396b0 msgr2=0x7fa6c003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.275+0000 7fa6e1a00700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6c00396b0 0x7fa6c003bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4006fd0 tx=0x7fa6c4006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.275+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 msgr2=0x7fa6dc19e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.275+0000 7fa6e1a00700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc19e470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0116a0 tx=0x7fa6cc011c10 comp rx=0 tx=0).stop 2024-04-17T07:47:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.275+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 shutdown_connections 2024-04-17T07:47:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.275+0000 7fa6e1a00700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6c00396b0 0x7fa6c003bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.275+0000 7fa6e1a00700 1 --2- 172.21.15.46:0/745822404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6dc07dde0 0x7fa6dc19e470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.276+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 >> 172.21.15.46:0/745822404 conn(0x7fa6dc1039b0 msgr2=0x7fa6dc1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.276+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 shutdown_connections 2024-04-17T07:47:30.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:30.276+0000 7fa6e1a00700 1 -- 172.21.15.46:0/745822404 wait complete. 2024-04-17T07:47:32.546 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.483+0000 7fa947e00700 1 Processor -- start 2024-04-17T07:47:32.546 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.483+0000 7fa947e00700 1 -- start start 2024-04-17T07:47:32.546 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.484+0000 7fa947e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94810a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:32.546 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.484+0000 7fa947e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa94810ad80 con 0x7fa94810a390 2024-04-17T07:47:32.546 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.484+0000 7fa946a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94810a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.484+0000 7fa946a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94810a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48084/0 (socket says 172.21.15.46:48084) 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.484+0000 7fa946a00700 1 -- 172.21.15.46:0/2534159548 learned_addr learned my addr 172.21.15.46:0/2534159548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.485+0000 7fa946a00700 1 -- 172.21.15.46:0/2534159548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94810b540 con 0x7fa94810a390 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.485+0000 7fa946a00700 1 --2- 172.21.15.46:0/2534159548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94810a7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9380098a0 tx=0x7fa938009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2136838a7fbe1a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.485+0000 7fa946000700 1 -- 172.21.15.46:0/2534159548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa938004160 con 0x7fa94810a390 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.485+0000 7fa946000700 1 -- 172.21.15.46:0/2534159548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa938004300 con 0x7fa94810a390 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.485+0000 7fa946000700 1 -- 172.21.15.46:0/2534159548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa938011b20 con 0x7fa94810a390 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.486+0000 7fa947e00700 1 -- 172.21.15.46:0/2534159548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 msgr2=0x7fa94810a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.486+0000 7fa947e00700 1 --2- 172.21.15.46:0/2534159548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94810a7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9380098a0 tx=0x7fa938009bb0 comp rx=0 tx=0).stop 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.486+0000 7fa947e00700 1 -- 172.21.15.46:0/2534159548 shutdown_connections 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.486+0000 7fa947e00700 1 --2- 172.21.15.46:0/2534159548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94810a7b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.486+0000 7fa947e00700 1 -- 172.21.15.46:0/2534159548 >> 172.21.15.46:0/2534159548 conn(0x7fa94807d890 msgr2=0x7fa948108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.486+0000 7fa947e00700 1 -- 172.21.15.46:0/2534159548 shutdown_connections 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.486+0000 7fa947e00700 1 -- 172.21.15.46:0/2534159548 wait complete. 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.487+0000 7fa947e00700 1 Processor -- start 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.488+0000 7fa947e00700 1 -- start start 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.488+0000 7fa947e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.488+0000 7fa947e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa94810e8f0 con 0x7fa94810a390 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.488+0000 7fa946a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.488+0000 7fa946a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94819e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48092/0 (socket says 172.21.15.46:48092) 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.488+0000 7fa946a00700 1 -- 172.21.15.46:0/758121903 learned_addr learned my addr 172.21.15.46:0/758121903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.489+0000 7fa946a00700 1 -- 172.21.15.46:0/758121903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94819e830 con 0x7fa94810a390 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.489+0000 7fa946a00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94819e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9380116a0 tx=0x7fa938011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.489+0000 7fa94ca00700 1 -- 172.21.15.46:0/758121903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa938003a40 con 0x7fa94810a390 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.489+0000 7fa94ca00700 1 -- 172.21.15.46:0/758121903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa938003be0 con 0x7fa94810a390 2024-04-17T07:47:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.489+0000 7fa94ca00700 1 -- 172.21.15.46:0/758121903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa93801a720 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.489+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94819eac0 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.490+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94819ef20 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.491+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9480640b0 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.491+0000 7fa94ca00700 1 -- 172.21.15.46:0/758121903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa938024aa0 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.491+0000 7fa94ca00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa92c0396b0 0x7fa92c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.491+0000 7fa94ca00700 1 -- 172.21.15.46:0/758121903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa93804de20 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.492+0000 7fa93fe00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa92c0396b0 0x7fa92c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.492+0000 7fa93fe00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa92c0396b0 0x7fa92c03bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa930006fd0 tx=0x7fa930006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.497+0000 7fa94ca00700 1 -- 172.21.15.46:0/758121903 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa938011e20 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.850+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa9481a18f0 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.854+0000 7fa94ca00700 1 -- 172.21.15.46:0/758121903 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fa938016df0 con 0x7fa94810a390 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.857+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa92c0396b0 msgr2=0x7fa92c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.857+0000 7fa947e00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa92c0396b0 0x7fa92c03bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa930006fd0 tx=0x7fa930006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.857+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 msgr2=0x7fa94819e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:32.549 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.857+0000 7fa947e00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94819e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9380116a0 tx=0x7fa938011ba0 comp rx=0 tx=0).stop 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.858+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 shutdown_connections 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.858+0000 7fa947e00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa92c0396b0 0x7fa92c03bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.858+0000 7fa947e00700 1 --2- 172.21.15.46:0/758121903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94810a390 0x7fa94819e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.858+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 >> 172.21.15.46:0/758121903 conn(0x7fa94807d890 msgr2=0x7fa948107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.858+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 shutdown_connections 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-04-17T07:47:31.858+0000 7fa947e00700 1 -- 172.21.15.46:0/758121903 wait complete. 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.550 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:32.551 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-04-17T07:47:32.650 INFO:tasks.cephadm:Fetching config... 2024-04-17T07:47:32.651 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:47:32.651 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-17T07:47:32.739 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-17T07:47:32.739 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:47:32.739 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-17T07:47:32.833 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-17T07:47:32.833 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:47:32.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-17T07:47:32.930 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-17T07:47:32.930 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:47:32.930 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-17T07:47:32.984 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-17T07:47:32.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2nUp2BhXlccuNtfzvbeZ5qNHV5sASbI0+ojkcm9ZfRE7R1JfVKa7HIMulm8U7qWVuQQYgRV+VjWSPolO41/ca6ZkhTUumhvGAjclMQ95LXJwtHYPSJ+/w25dfREDxtyj6FWO2i+01EuqEeusWWeG1vWGAp4ee/JUl2Nc7xDbRqqqgKxthyZHyvR6XmpoQ3F3DwRI3Ojli031Z+hF6rZZ3n3kugfc/1+FF9EwUr4z/nL5kZCmyURZvrKfza7oe/VYjX5XecpqJ2kFlZyBMsJQOobRuZXgs5PrF42HLwUwNXIpOv/cJVCnFJsTPXnE4wysma8XFcCpvOKejXaV71yfHWFxeQYYYABDLJB1M4l9JFgWDcmdDkj3GGq/ZEgpKuI57WAZ/0fuqQCYVCZ0U7qN7N6j2YOmjNT0p6qcWa4+HAGow+52K86no6TQLXBKakMGR2Qu5ZopDHsojeG9nFtWDWLzsuRpTsuUj26T32DS7DnXcKScL8v/RYiCQDx3rsZk= ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T07:47:33.084 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:32 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/758121903' entity='client.admin' 2024-04-17T07:47:33.105 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2nUp2BhXlccuNtfzvbeZ5qNHV5sASbI0+ojkcm9ZfRE7R1JfVKa7HIMulm8U7qWVuQQYgRV+VjWSPolO41/ca6ZkhTUumhvGAjclMQ95LXJwtHYPSJ+/w25dfREDxtyj6FWO2i+01EuqEeusWWeG1vWGAp4ee/JUl2Nc7xDbRqqqgKxthyZHyvR6XmpoQ3F3DwRI3Ojli031Z+hF6rZZ3n3kugfc/1+FF9EwUr4z/nL5kZCmyURZvrKfza7oe/VYjX5XecpqJ2kFlZyBMsJQOobRuZXgs5PrF42HLwUwNXIpOv/cJVCnFJsTPXnE4wysma8XFcCpvOKejXaV71yfHWFxeQYYYABDLJB1M4l9JFgWDcmdDkj3GGq/ZEgpKuI57WAZ/0fuqQCYVCZ0U7qN7N6j2YOmjNT0p6qcWa4+HAGow+52K86no6TQLXBKakMGR2Qu5ZopDHsojeG9nFtWDWLzsuRpTsuUj26T32DS7DnXcKScL8v/RYiCQDx3rsZk= ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:47:33.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2nUp2BhXlccuNtfzvbeZ5qNHV5sASbI0+ojkcm9ZfRE7R1JfVKa7HIMulm8U7qWVuQQYgRV+VjWSPolO41/ca6ZkhTUumhvGAjclMQ95LXJwtHYPSJ+/w25dfREDxtyj6FWO2i+01EuqEeusWWeG1vWGAp4ee/JUl2Nc7xDbRqqqgKxthyZHyvR6XmpoQ3F3DwRI3Ojli031Z+hF6rZZ3n3kugfc/1+FF9EwUr4z/nL5kZCmyURZvrKfza7oe/VYjX5XecpqJ2kFlZyBMsJQOobRuZXgs5PrF42HLwUwNXIpOv/cJVCnFJsTPXnE4wysma8XFcCpvOKejXaV71yfHWFxeQYYYABDLJB1M4l9JFgWDcmdDkj3GGq/ZEgpKuI57WAZ/0fuqQCYVCZ0U7qN7N6j2YOmjNT0p6qcWa4+HAGow+52K86no6TQLXBKakMGR2Qu5ZopDHsojeG9nFtWDWLzsuRpTsuUj26T32DS7DnXcKScL8v/RYiCQDx3rsZk= ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T07:47:33.210 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2nUp2BhXlccuNtfzvbeZ5qNHV5sASbI0+ojkcm9ZfRE7R1JfVKa7HIMulm8U7qWVuQQYgRV+VjWSPolO41/ca6ZkhTUumhvGAjclMQ95LXJwtHYPSJ+/w25dfREDxtyj6FWO2i+01EuqEeusWWeG1vWGAp4ee/JUl2Nc7xDbRqqqgKxthyZHyvR6XmpoQ3F3DwRI3Ojli031Z+hF6rZZ3n3kugfc/1+FF9EwUr4z/nL5kZCmyURZvrKfza7oe/VYjX5XecpqJ2kFlZyBMsJQOobRuZXgs5PrF42HLwUwNXIpOv/cJVCnFJsTPXnE4wysma8XFcCpvOKejXaV71yfHWFxeQYYYABDLJB1M4l9JFgWDcmdDkj3GGq/ZEgpKuI57WAZ/0fuqQCYVCZ0U7qN7N6j2YOmjNT0p6qcWa4+HAGow+52K86no6TQLXBKakMGR2Qu5ZopDHsojeG9nFtWDWLzsuRpTsuUj26T32DS7DnXcKScL8v/RYiCQDx3rsZk= ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:47:33.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2nUp2BhXlccuNtfzvbeZ5qNHV5sASbI0+ojkcm9ZfRE7R1JfVKa7HIMulm8U7qWVuQQYgRV+VjWSPolO41/ca6ZkhTUumhvGAjclMQ95LXJwtHYPSJ+/w25dfREDxtyj6FWO2i+01EuqEeusWWeG1vWGAp4ee/JUl2Nc7xDbRqqqgKxthyZHyvR6XmpoQ3F3DwRI3Ojli031Z+hF6rZZ3n3kugfc/1+FF9EwUr4z/nL5kZCmyURZvrKfza7oe/VYjX5XecpqJ2kFlZyBMsJQOobRuZXgs5PrF42HLwUwNXIpOv/cJVCnFJsTPXnE4wysma8XFcCpvOKejXaV71yfHWFxeQYYYABDLJB1M4l9JFgWDcmdDkj3GGq/ZEgpKuI57WAZ/0fuqQCYVCZ0U7qN7N6j2YOmjNT0p6qcWa4+HAGow+52K86no6TQLXBKakMGR2Qu5ZopDHsojeG9nFtWDWLzsuRpTsuUj26T32DS7DnXcKScL8v/RYiCQDx3rsZk= ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T07:47:33.282 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2nUp2BhXlccuNtfzvbeZ5qNHV5sASbI0+ojkcm9ZfRE7R1JfVKa7HIMulm8U7qWVuQQYgRV+VjWSPolO41/ca6ZkhTUumhvGAjclMQ95LXJwtHYPSJ+/w25dfREDxtyj6FWO2i+01EuqEeusWWeG1vWGAp4ee/JUl2Nc7xDbRqqqgKxthyZHyvR6XmpoQ3F3DwRI3Ojli031Z+hF6rZZ3n3kugfc/1+FF9EwUr4z/nL5kZCmyURZvrKfza7oe/VYjX5XecpqJ2kFlZyBMsJQOobRuZXgs5PrF42HLwUwNXIpOv/cJVCnFJsTPXnE4wysma8XFcCpvOKejXaV71yfHWFxeQYYYABDLJB1M4l9JFgWDcmdDkj3GGq/ZEgpKuI57WAZ/0fuqQCYVCZ0U7qN7N6j2YOmjNT0p6qcWa4+HAGow+52K86no6TQLXBKakMGR2Qu5ZopDHsojeG9nFtWDWLzsuRpTsuUj26T32DS7DnXcKScL8v/RYiCQDx3rsZk= ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:47:33.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-17T07:47:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:47:36.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.843+0000 7f51a1800700 1 -- 172.21.15.46:0/1788948543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 msgr2=0x7f519c105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:36.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.843+0000 7f51a1800700 1 --2- 172.21.15.46:0/1788948543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c105330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f518c009910 tx=0x7f518c009c20 comp rx=0 tx=0).stop 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.843+0000 7f51a1800700 1 -- 172.21.15.46:0/1788948543 shutdown_connections 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.843+0000 7f51a1800700 1 --2- 172.21.15.46:0/1788948543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c105330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.843+0000 7f51a1800700 1 -- 172.21.15.46:0/1788948543 >> 172.21.15.46:0/1788948543 conn(0x7f519c0fc950 msgr2=0x7f519c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.843+0000 7f51a1800700 1 -- 172.21.15.46:0/1788948543 shutdown_connections 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.843+0000 7f51a1800700 1 -- 172.21.15.46:0/1788948543 wait complete. 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.844+0000 7f51a1800700 1 Processor -- start 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.845+0000 7f51a1800700 1 -- start start 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.845+0000 7f51a1800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.845+0000 7f51a1800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1084e0 con 0x7f519c102f40 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.845+0000 7f519aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.845+0000 7f519aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48108/0 (socket says 172.21.15.46:48108) 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.845+0000 7f519aa00700 1 -- 172.21.15.46:0/2578400152 learned_addr learned my addr 172.21.15.46:0/2578400152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.846+0000 7f519aa00700 1 -- 172.21.15.46:0/2578400152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c19a290 con 0x7f519c102f40 2024-04-17T07:47:36.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.846+0000 7f519aa00700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c199d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f518c005950 tx=0x7f518c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.846+0000 7f5193400700 1 -- 172.21.15.46:0/2578400152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f518c026070 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.846+0000 7f5193400700 1 -- 172.21.15.46:0/2578400152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518c00bbd0 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.846+0000 7f5193400700 1 -- 172.21.15.46:0/2578400152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f518c003ad0 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.846+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c19a580 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.846+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c19d210 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.848+0000 7f5193400700 1 -- 172.21.15.46:0/2578400152 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f518c029070 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.848+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c108c60 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.848+0000 7f5193400700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5180039700 0x7f518003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.848+0000 7f5193400700 1 -- 172.21.15.46:0/2578400152 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f518c01e070 con 0x7f519c102f40 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.848+0000 7f519a000700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5180039700 0x7f518003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.849+0000 7f519a000700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5180039700 0x7f518003bbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5188006fd0 tx=0x7f5188006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:36.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:35.854+0000 7f5193400700 1 -- 172.21.15.46:0/2578400152 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f518c013070 con 0x7f519c102f40 2024-04-17T07:47:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.109+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f519c02e180 con 0x7f519c102f40 2024-04-17T07:47:36.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.115+0000 7f5193400700 1 -- 172.21.15.46:0/2578400152 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f518c0300c0 con 0x7f519c102f40 2024-04-17T07:47:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5180039700 msgr2=0x7f518003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5180039700 0x7f518003bbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5188006fd0 tx=0x7f5188006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 msgr2=0x7f519c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c199d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f518c005950 tx=0x7f518c005030 comp rx=0 tx=0).stop 2024-04-17T07:47:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 shutdown_connections 2024-04-17T07:47:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5180039700 0x7f518003bbc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 --2- 172.21.15.46:0/2578400152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f519c102f40 0x7f519c199d50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 >> 172.21.15.46:0/2578400152 conn(0x7f519c0fc950 msgr2=0x7f519c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.118+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 shutdown_connections 2024-04-17T07:47:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:36.119+0000 7f51a1800700 1 -- 172.21.15.46:0/2578400152 wait complete. 2024-04-17T07:47:36.834 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-17T07:47:36.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-17T07:47:37.319 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:37 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2578400152' entity='client.admin' 2024-04-17T07:47:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:47:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.605+0000 7f9c31c00700 1 -- 172.21.15.46:0/525491388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 msgr2=0x7f9c2c102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.605+0000 7f9c31c00700 1 --2- 172.21.15.46:0/525491388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c102720 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009910 tx=0x7f9c1c009c20 comp rx=0 tx=0).stop 2024-04-17T07:47:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.606+0000 7f9c31c00700 1 -- 172.21.15.46:0/525491388 shutdown_connections 2024-04-17T07:47:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.606+0000 7f9c31c00700 1 --2- 172.21.15.46:0/525491388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c102720 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.606+0000 7f9c31c00700 1 -- 172.21.15.46:0/525491388 >> 172.21.15.46:0/525491388 conn(0x7f9c2c0fd900 msgr2=0x7f9c2c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.606+0000 7f9c31c00700 1 -- 172.21.15.46:0/525491388 shutdown_connections 2024-04-17T07:47:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.606+0000 7f9c31c00700 1 -- 172.21.15.46:0/525491388 wait complete. 2024-04-17T07:47:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.607+0000 7f9c31c00700 1 Processor -- start 2024-04-17T07:47:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.607+0000 7f9c31c00700 1 -- start start 2024-04-17T07:47:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.607+0000 7f9c31c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.607+0000 7f9c31c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c108390 con 0x7f9c2c102300 2024-04-17T07:47:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.608+0000 7f9c2be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.608+0000 7f9c2be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32856/0 (socket says 172.21.15.46:32856) 2024-04-17T07:47:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.608+0000 7f9c2be00700 1 -- 172.21.15.46:0/4213321486 learned_addr learned my addr 172.21.15.46:0/4213321486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.608+0000 7f9c2be00700 1 -- 172.21.15.46:0/4213321486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c19a170 con 0x7f9c2c102300 2024-04-17T07:47:39.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.608+0000 7f9c2be00700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c199c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c005850 tx=0x7f9c1c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:39.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.609+0000 7f9c28c00700 1 -- 172.21.15.46:0/4213321486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c1c026070 con 0x7f9c2c102300 2024-04-17T07:47:39.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.609+0000 7f9c28c00700 1 -- 172.21.15.46:0/4213321486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c1c0044a0 con 0x7f9c2c102300 2024-04-17T07:47:39.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.609+0000 7f9c28c00700 1 -- 172.21.15.46:0/4213321486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c1c003ab0 con 0x7f9c2c102300 2024-04-17T07:47:39.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.609+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2c19a400 con 0x7f9c2c102300 2024-04-17T07:47:39.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.609+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2c19d0f0 con 0x7f9c2c102300 2024-04-17T07:47:39.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.610+0000 7f9c28c00700 1 -- 172.21.15.46:0/4213321486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9c1c029070 con 0x7f9c2c102300 2024-04-17T07:47:39.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.610+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2c108b10 con 0x7f9c2c102300 2024-04-17T07:47:39.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.611+0000 7f9c28c00700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c100396b0 0x7f9c1003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:39.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.611+0000 7f9c28c00700 1 -- 172.21.15.46:0/4213321486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9c1c01e070 con 0x7f9c2c102300 2024-04-17T07:47:39.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.611+0000 7f9c2b400700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c100396b0 0x7f9c1003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:39.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.612+0000 7f9c2b400700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c100396b0 0x7f9c1003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006fd0 tx=0x7f9c18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:39.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.616+0000 7f9c28c00700 1 -- 172.21.15.46:0/4213321486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9c1c013070 con 0x7f9c2c102300 2024-04-17T07:47:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.860+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f9c2c02e130 con 0x7f9c100396b0 2024-04-17T07:47:39.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.866+0000 7f9c28c00700 1 -- 172.21.15.46:0/4213321486 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c2c02e130 con 0x7f9c100396b0 2024-04-17T07:47:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c100396b0 msgr2=0x7f9c1003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c100396b0 0x7f9c1003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006fd0 tx=0x7f9c18006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 msgr2=0x7f9c2c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c199c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c005850 tx=0x7f9c1c005030 comp rx=0 tx=0).stop 2024-04-17T07:47:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 shutdown_connections 2024-04-17T07:47:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c100396b0 0x7f9c1003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 --2- 172.21.15.46:0/4213321486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c2c102300 0x7f9c2c199c30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 >> 172.21.15.46:0/4213321486 conn(0x7f9c2c0fd900 msgr2=0x7f9c2c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.869+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 shutdown_connections 2024-04-17T07:47:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:39.870+0000 7f9c31c00700 1 -- 172.21.15.46:0/4213321486 wait complete. 2024-04-17T07:47:40.650 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-04-17T07:47:40.650 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:47:40.650 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-04-17T07:47:40.669 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:47:40.670 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:47:40.727 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-04-17T07:47:40.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch host add smithi053 2024-04-17T07:47:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:47:41.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:40 smithi046 ceph-mon[32038]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-17T07:47:41.993 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:41 smithi046 ceph-mon[32038]: Updating smithi046:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:47:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.024+0000 7f7365800700 1 -- 172.21.15.46:0/959474300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 msgr2=0x7f73601088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.024+0000 7f7365800700 1 --2- 172.21.15.46:0/959474300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f73601088d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f735000b0a0 tx=0x7f735000b3b0 comp rx=0 tx=0).stop 2024-04-17T07:47:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.024+0000 7f7365800700 1 -- 172.21.15.46:0/959474300 shutdown_connections 2024-04-17T07:47:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.024+0000 7f7365800700 1 --2- 172.21.15.46:0/959474300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f73601088d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.024+0000 7f7365800700 1 -- 172.21.15.46:0/959474300 >> 172.21.15.46:0/959474300 conn(0x7f7360076eb0 msgr2=0x7f73600772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.024+0000 7f7365800700 1 -- 172.21.15.46:0/959474300 shutdown_connections 2024-04-17T07:47:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.024+0000 7f7365800700 1 -- 172.21.15.46:0/959474300 wait complete. 2024-04-17T07:47:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.025+0000 7f7365800700 1 Processor -- start 2024-04-17T07:47:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.025+0000 7f7365800700 1 -- start start 2024-04-17T07:47:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.026+0000 7f7365800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f7360197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.026+0000 7f7365800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f736010ca80 con 0x7f73601084f0 2024-04-17T07:47:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.026+0000 7f735ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f7360197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.026+0000 7f735ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f7360197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32870/0 (socket says 172.21.15.46:32870) 2024-04-17T07:47:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.026+0000 7f735ea00700 1 -- 172.21.15.46:0/2743830045 learned_addr learned my addr 172.21.15.46:0/2743830045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.026+0000 7f735ea00700 1 -- 172.21.15.46:0/2743830045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73601983f0 con 0x7f73601084f0 2024-04-17T07:47:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.027+0000 7f735ea00700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f7360197eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7350015040 tx=0x7f7350003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.027+0000 7f7357e00700 1 -- 172.21.15.46:0/2743830045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7350017040 con 0x7f73601084f0 2024-04-17T07:47:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.027+0000 7f7357e00700 1 -- 172.21.15.46:0/2743830045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7350022070 con 0x7f73601084f0 2024-04-17T07:47:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.027+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7360198680 con 0x7f73601084f0 2024-04-17T07:47:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.027+0000 7f7357e00700 1 -- 172.21.15.46:0/2743830045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f735001d5d0 con 0x7f73601084f0 2024-04-17T07:47:43.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.027+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736019c780 con 0x7f73601084f0 2024-04-17T07:47:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.028+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7360103ba0 con 0x7f73601084f0 2024-04-17T07:47:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.028+0000 7f7357e00700 1 -- 172.21.15.46:0/2743830045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7350004000 con 0x7f73601084f0 2024-04-17T07:47:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.029+0000 7f7357e00700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7344039650 0x7f734403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.029+0000 7f7357e00700 1 -- 172.21.15.46:0/2743830045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f735000f050 con 0x7f73601084f0 2024-04-17T07:47:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.030+0000 7f7356000700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7344039650 0x7f734403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:43.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.030+0000 7f7356000700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7344039650 0x7f734403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7348006fd0 tx=0x7f7348006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.035+0000 7f7357e00700 1 -- 172.21.15.46:0/2743830045 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7350007e20 con 0x7f73601084f0 2024-04-17T07:47:43.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:42 smithi046 ceph-mon[32038]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:43.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:42 smithi046 ceph-mon[32038]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:47:43.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:42 smithi046 ceph-mon[32038]: Updating smithi046:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:47:43.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:43.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:43.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:43.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:43.269+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) -- 0x7f7360199210 con 0x7f7344039650 2024-04-17T07:47:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:47:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:47:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:47:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:45.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:44 smithi046 ceph-mon[32038]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:47:45.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:44 smithi046 ceph-mon[32038]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:45.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:44 smithi046 ceph-mon[32038]: Deploying cephadm binary to smithi053 2024-04-17T07:47:47.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:46 smithi046 ceph-mon[32038]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:48.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.360+0000 7f7357e00700 1 -- 172.21.15.46:0/2743830045 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f7360199210 con 0x7f7344039650 2024-04-17T07:47:48.361 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7344039650 msgr2=0x7f734403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7344039650 0x7f734403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7348006fd0 tx=0x7f7348006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 msgr2=0x7f7360197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f7360197eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7350015040 tx=0x7f7350003d50 comp rx=0 tx=0).stop 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 shutdown_connections 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7344039650 0x7f734403bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 --2- 172.21.15.46:0/2743830045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73601084f0 0x7f7360197eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 >> 172.21.15.46:0/2743830045 conn(0x7f7360076eb0 msgr2=0x7f73600fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.364+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 shutdown_connections 2024-04-17T07:47:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:48.365+0000 7f7365800700 1 -- 172.21.15.46:0/2743830045 wait complete. 2024-04-17T07:47:49.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:48 smithi046 ceph-mon[32038]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:49.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:48 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:49.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:48 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:47:49.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:48 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:49.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch host ls --format=json 2024-04-17T07:47:49.734 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:47:50.188 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:49 smithi046 ceph-mon[32038]: Added host smithi053 2024-04-17T07:47:50.188 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:51.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:50 smithi046 ceph-mon[32038]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.736+0000 7ff472000700 1 -- 172.21.15.46:0/3088149948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 msgr2=0x7ff46c0751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.736+0000 7ff472000700 1 --2- 172.21.15.46:0/3088149948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c0751d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff460009910 tx=0x7ff460009c20 comp rx=0 tx=0).stop 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.736+0000 7ff472000700 1 -- 172.21.15.46:0/3088149948 shutdown_connections 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.736+0000 7ff472000700 1 --2- 172.21.15.46:0/3088149948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c0751d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.736+0000 7ff472000700 1 -- 172.21.15.46:0/3088149948 >> 172.21.15.46:0/3088149948 conn(0x7ff46c0fddb0 msgr2=0x7ff46c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.736+0000 7ff472000700 1 -- 172.21.15.46:0/3088149948 shutdown_connections 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.736+0000 7ff472000700 1 -- 172.21.15.46:0/3088149948 wait complete. 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.737+0000 7ff472000700 1 Processor -- start 2024-04-17T07:47:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.737+0000 7ff472000700 1 -- start start 2024-04-17T07:47:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.737+0000 7ff472000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.737+0000 7ff472000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c112f40 con 0x7ff46c074df0 2024-04-17T07:47:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.738+0000 7ff470c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.738+0000 7ff470c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c19e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51556/0 (socket says 172.21.15.46:51556) 2024-04-17T07:47:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.738+0000 7ff470c00700 1 -- 172.21.15.46:0/238446439 learned_addr learned my addr 172.21.15.46:0/238446439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.738+0000 7ff470c00700 1 -- 172.21.15.46:0/238446439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c19e9e0 con 0x7ff46c074df0 2024-04-17T07:47:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.739+0000 7ff470c00700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c19e4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff460006010 tx=0x7ff460005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.739+0000 7ff469600700 1 -- 172.21.15.46:0/238446439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff460026070 con 0x7ff46c074df0 2024-04-17T07:47:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.739+0000 7ff469600700 1 -- 172.21.15.46:0/238446439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff46000bc30 con 0x7ff46c074df0 2024-04-17T07:47:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.739+0000 7ff469600700 1 -- 172.21.15.46:0/238446439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4600039a0 con 0x7ff46c074df0 2024-04-17T07:47:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.739+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c19ec70 con 0x7ff46c074df0 2024-04-17T07:47:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.739+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c19f0d0 con 0x7ff46c074df0 2024-04-17T07:47:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.740+0000 7ff469600700 1 -- 172.21.15.46:0/238446439 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff460029070 con 0x7ff46c074df0 2024-04-17T07:47:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.740+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c076530 con 0x7ff46c074df0 2024-04-17T07:47:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.741+0000 7ff469600700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4540396b0 0x7ff45403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.741+0000 7ff469600700 1 -- 172.21.15.46:0/238446439 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff46001e070 con 0x7ff46c074df0 2024-04-17T07:47:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.741+0000 7ff46be00700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4540396b0 0x7ff45403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.742+0000 7ff46be00700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4540396b0 0x7ff45403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff458006fd0 tx=0x7ff458006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:51.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.747+0000 7ff469600700 1 -- 172.21.15.46:0/238446439 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff460013070 con 0x7ff46c074df0 2024-04-17T07:47:51.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.985+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff46c19f800 con 0x7ff4540396b0 2024-04-17T07:47:51.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.987+0000 7ff469600700 1 -- 172.21.15.46:0/238446439 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7ff46c19f800 con 0x7ff4540396b0 2024-04-17T07:47:51.987 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:47:51.987 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-04-17T07:47:51.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4540396b0 msgr2=0x7ff45403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4540396b0 0x7ff45403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff458006fd0 tx=0x7ff458006e40 comp rx=0 tx=0).stop 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 msgr2=0x7ff46c19e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c19e4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff460006010 tx=0x7ff460005030 comp rx=0 tx=0).stop 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 shutdown_connections 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4540396b0 0x7ff45403bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 --2- 172.21.15.46:0/238446439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff46c074df0 0x7ff46c19e4a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.990+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 >> 172.21.15.46:0/238446439 conn(0x7ff46c0fddb0 msgr2=0x7ff46c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.991+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 shutdown_connections 2024-04-17T07:47:51.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:51.991+0000 7ff472000700 1 -- 172.21.15.46:0/238446439 wait complete. 2024-04-17T07:47:52.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:52 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:52.728 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi062 2024-04-17T07:47:52.728 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:47:52.728 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.conf 2024-04-17T07:47:52.748 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:47:52.748 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:47:52.806 INFO:tasks.cephadm:Adding host smithi062 to orchestrator... 2024-04-17T07:47:52.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch host add smithi062 2024-04-17T07:47:53.248 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:47:53.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:53 smithi046 ceph-mon[32038]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:53.408 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:53 smithi046 ceph-mon[32038]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.291+0000 7fb7dfc00700 1 -- 172.21.15.46:0/3551062855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 msgr2=0x7fb7d80751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.291+0000 7fb7dfc00700 1 --2- 172.21.15.46:0/3551062855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d80751d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009910 tx=0x7fb7cc009c20 comp rx=0 tx=0).stop 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.292+0000 7fb7dfc00700 1 -- 172.21.15.46:0/3551062855 shutdown_connections 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.292+0000 7fb7dfc00700 1 --2- 172.21.15.46:0/3551062855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d80751d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.292+0000 7fb7dfc00700 1 -- 172.21.15.46:0/3551062855 >> 172.21.15.46:0/3551062855 conn(0x7fb7d80fdf90 msgr2=0x7fb7d81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.292+0000 7fb7dfc00700 1 -- 172.21.15.46:0/3551062855 shutdown_connections 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.292+0000 7fb7dfc00700 1 -- 172.21.15.46:0/3551062855 wait complete. 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.293+0000 7fb7dfc00700 1 Processor -- start 2024-04-17T07:47:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.293+0000 7fb7dfc00700 1 -- start start 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.293+0000 7fb7dfc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.293+0000 7fb7dfc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8113100 con 0x7fb7d8074df0 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.294+0000 7fb7dd200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.294+0000 7fb7dd200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51570/0 (socket says 172.21.15.46:51570) 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.294+0000 7fb7dd200700 1 -- 172.21.15.46:0/1875145243 learned_addr learned my addr 172.21.15.46:0/1875145243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.294+0000 7fb7dd200700 1 -- 172.21.15.46:0/1875145243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d819eb10 con 0x7fb7d8074df0 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.294+0000 7fb7dd200700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d819e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006010 tx=0x7fb7cc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.295+0000 7fb7d5600700 1 -- 172.21.15.46:0/1875145243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7cc026070 con 0x7fb7d8074df0 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.295+0000 7fb7d5600700 1 -- 172.21.15.46:0/1875145243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc00bc30 con 0x7fb7d8074df0 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.295+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d819eda0 con 0x7fb7d8074df0 2024-04-17T07:47:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.295+0000 7fb7d5600700 1 -- 172.21.15.46:0/1875145243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7cc0039a0 con 0x7fb7d8074df0 2024-04-17T07:47:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.295+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d819f200 con 0x7fb7d8074df0 2024-04-17T07:47:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.296+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d8076530 con 0x7fb7d8074df0 2024-04-17T07:47:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.296+0000 7fb7d5600700 1 -- 172.21.15.46:0/1875145243 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb7cc029070 con 0x7fb7d8074df0 2024-04-17T07:47:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.297+0000 7fb7d5600700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb7c00396b0 0x7fb7c003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:47:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.297+0000 7fb7d5600700 1 -- 172.21.15.46:0/1875145243 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7cc01e070 con 0x7fb7d8074df0 2024-04-17T07:47:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.297+0000 7fb7d7e00700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb7c00396b0 0x7fb7c003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:47:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.298+0000 7fb7d7e00700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb7c00396b0 0x7fb7c003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4006fd0 tx=0x7fb7c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:47:55.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.303+0000 7fb7d5600700 1 -- 172.21.15.46:0/1875145243 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb7cc013070 con 0x7fb7d8074df0 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:47:55.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:47:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:47:55.539+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}) -- 0x7fb7d819f930 con 0x7fb7c00396b0 2024-04-17T07:47:56.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:56 smithi046 ceph-mon[32038]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-17T07:47:56.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:56 smithi046 ceph-mon[32038]: Updating smithi053:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:47:56.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:56 smithi046 ceph-mon[32038]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:47:56.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:56 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:56.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:56 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:56.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:56 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:47:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:57 smithi046 ceph-mon[32038]: Updating smithi053:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:47:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:57 smithi046 ceph-mon[32038]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:47:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:57 smithi046 ceph-mon[32038]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:47:58.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:58 smithi046 ceph-mon[32038]: Deploying cephadm binary to smithi062 2024-04-17T07:47:59.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:47:59 smithi046 ceph-mon[32038]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:00.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.610+0000 7fb7d5600700 1 -- 172.21.15.46:0/1875145243 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb7d819f930 con 0x7fb7c00396b0 2024-04-17T07:48:00.610 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi062' with addr '172.21.15.62' 2024-04-17T07:48:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb7c00396b0 msgr2=0x7fb7c003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb7c00396b0 0x7fb7c003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4006fd0 tx=0x7fb7c4006e40 comp rx=0 tx=0).stop 2024-04-17T07:48:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 msgr2=0x7fb7d819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d819e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006010 tx=0x7fb7cc005030 comp rx=0 tx=0).stop 2024-04-17T07:48:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 shutdown_connections 2024-04-17T07:48:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb7c00396b0 0x7fb7c003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 --2- 172.21.15.46:0/1875145243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d8074df0 0x7fb7d819e5d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 >> 172.21.15.46:0/1875145243 conn(0x7fb7d80fdf90 msgr2=0x7fb7d80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 shutdown_connections 2024-04-17T07:48:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:00.614+0000 7fb7dfc00700 1 -- 172.21.15.46:0/1875145243 wait complete. 2024-04-17T07:48:01.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:01 smithi046 ceph-mon[32038]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:01.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:01 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:01.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:01 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:01.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch host ls --format=json 2024-04-17T07:48:01.780 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:48:02.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:02 smithi046 ceph-mon[32038]: Added host smithi062 2024-04-17T07:48:02.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:02.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:03.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:03 smithi046 ceph-mon[32038]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.659+0000 7f495e600700 1 -- 172.21.15.46:0/3629136101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 msgr2=0x7f495810aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.659+0000 7f495e600700 1 --2- 172.21.15.46:0/3629136101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495810aa70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4948009910 tx=0x7f4948009c20 comp rx=0 tx=0).stop 2024-04-17T07:48:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.659+0000 7f495e600700 1 -- 172.21.15.46:0/3629136101 shutdown_connections 2024-04-17T07:48:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.659+0000 7f495e600700 1 --2- 172.21.15.46:0/3629136101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495810aa70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.659+0000 7f495e600700 1 -- 172.21.15.46:0/3629136101 >> 172.21.15.46:0/3629136101 conn(0x7f4958100160 msgr2=0x7f4958102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.659+0000 7f495e600700 1 -- 172.21.15.46:0/3629136101 shutdown_connections 2024-04-17T07:48:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.659+0000 7f495e600700 1 -- 172.21.15.46:0/3629136101 wait complete. 2024-04-17T07:48:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.660+0000 7f495e600700 1 Processor -- start 2024-04-17T07:48:03.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.660+0000 7f495e600700 1 -- start start 2024-04-17T07:48:03.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.661+0000 7f495e600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:03.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.661+0000 7f495e600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f495810ecd0 con 0x7f495810a690 2024-04-17T07:48:03.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.661+0000 7f4957400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:03.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.661+0000 7f4957400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48906/0 (socket says 172.21.15.46:48906) 2024-04-17T07:48:03.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.661+0000 7f4957400700 1 -- 172.21.15.46:0/3428445611 learned_addr learned my addr 172.21.15.46:0/3428445611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:48:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.662+0000 7f4957400700 1 -- 172.21.15.46:0/3428445611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495819a6b0 con 0x7f495810a690 2024-04-17T07:48:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.662+0000 7f4957400700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495819a170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f4948005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.662+0000 7f494fe00700 1 -- 172.21.15.46:0/3428445611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4948026070 con 0x7f495810a690 2024-04-17T07:48:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.662+0000 7f494fe00700 1 -- 172.21.15.46:0/3428445611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f494800bbd0 con 0x7f495810a690 2024-04-17T07:48:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.662+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495819a9a0 con 0x7f495810a690 2024-04-17T07:48:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.662+0000 7f494fe00700 1 -- 172.21.15.46:0/3428445611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4948003910 con 0x7f495810a690 2024-04-17T07:48:03.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.662+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495819ea40 con 0x7f495810a690 2024-04-17T07:48:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.664+0000 7f494fe00700 1 -- 172.21.15.46:0/3428445611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4948029070 con 0x7f495810a690 2024-04-17T07:48:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.664+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4958105dd0 con 0x7f495810a690 2024-04-17T07:48:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.664+0000 7f494fe00700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f493c0396b0 0x7f493c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.664+0000 7f494fe00700 1 -- 172.21.15.46:0/3428445611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f494801e070 con 0x7f495810a690 2024-04-17T07:48:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.664+0000 7f4956a00700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f493c0396b0 0x7f493c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.665+0000 7f4956a00700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f493c0396b0 0x7f493c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4944006fd0 tx=0x7f4944006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.670+0000 7f494fe00700 1 -- 172.21.15.46:0/3428445611 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4948013070 con 0x7f495810a690 2024-04-17T07:48:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.908+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f495819b4d0 con 0x7f493c0396b0 2024-04-17T07:48:03.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.911+0000 7f494fe00700 1 -- 172.21.15.46:0/3428445611 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f495819b4d0 con 0x7f493c0396b0 2024-04-17T07:48:03.912 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:48:03.912 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}] 2024-04-17T07:48:03.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f493c0396b0 msgr2=0x7f493c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:03.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f493c0396b0 0x7f493c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4944006fd0 tx=0x7f4944006e40 comp rx=0 tx=0).stop 2024-04-17T07:48:03.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 msgr2=0x7f495819a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:03.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495819a170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f4948005030 comp rx=0 tx=0).stop 2024-04-17T07:48:03.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 shutdown_connections 2024-04-17T07:48:03.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f493c0396b0 0x7f493c03bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 --2- 172.21.15.46:0/3428445611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f495810a690 0x7f495819a170 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 >> 172.21.15.46:0/3428445611 conn(0x7f4958100160 msgr2=0x7f495810c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 shutdown_connections 2024-04-17T07:48:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:03.914+0000 7f495e600700 1 -- 172.21.15.46:0/3428445611 wait complete. 2024-04-17T07:48:04.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:04 smithi046 ceph-mon[32038]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:04.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:04.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:04 smithi046 ceph-mon[32038]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:48:04.680 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-17T07:48:04.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd crush tunables default 2024-04-17T07:48:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:48:06.586 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:06 smithi046 ceph-mon[32038]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:07.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.111+0000 7f874ec00700 1 -- 172.21.15.46:0/3782415075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 msgr2=0x7f8748069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:07.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.111+0000 7f874ec00700 1 --2- 172.21.15.46:0/3782415075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f8748069e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f873800b0a0 tx=0x7f873800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:48:07.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.111+0000 7f874ec00700 1 -- 172.21.15.46:0/3782415075 shutdown_connections 2024-04-17T07:48:07.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.111+0000 7f874ec00700 1 --2- 172.21.15.46:0/3782415075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f8748069e70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:07.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.111+0000 7f874ec00700 1 -- 172.21.15.46:0/3782415075 >> 172.21.15.46:0/3782415075 conn(0x7f8748076eb0 msgr2=0x7f87480772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:07.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.112+0000 7f874ec00700 1 -- 172.21.15.46:0/3782415075 shutdown_connections 2024-04-17T07:48:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.112+0000 7f874ec00700 1 -- 172.21.15.46:0/3782415075 wait complete. 2024-04-17T07:48:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.113+0000 7f874ec00700 1 Processor -- start 2024-04-17T07:48:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.113+0000 7f874ec00700 1 -- start start 2024-04-17T07:48:07.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.113+0000 7f874ec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f874819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.113+0000 7f874ec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8748113110 con 0x7f8748107390 2024-04-17T07:48:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.114+0000 7f8747e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f874819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.114+0000 7f8747e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f874819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48922/0 (socket says 172.21.15.46:48922) 2024-04-17T07:48:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.114+0000 7f8747e00700 1 -- 172.21.15.46:0/844934401 learned_addr learned my addr 172.21.15.46:0/844934401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:48:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.114+0000 7f8747e00700 1 -- 172.21.15.46:0/844934401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874819eb10 con 0x7f8748107390 2024-04-17T07:48:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.114+0000 7f8747e00700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f874819e5d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87380062a0 tx=0x7f8738003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.115+0000 7f8745600700 1 -- 172.21.15.46:0/844934401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8738017040 con 0x7f8748107390 2024-04-17T07:48:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.115+0000 7f8745600700 1 -- 172.21.15.46:0/844934401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8738022070 con 0x7f8748107390 2024-04-17T07:48:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.115+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874819eda0 con 0x7f8748107390 2024-04-17T07:48:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.115+0000 7f8745600700 1 -- 172.21.15.46:0/844934401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f873801d440 con 0x7f8748107390 2024-04-17T07:48:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.115+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874819f200 con 0x7f8748107390 2024-04-17T07:48:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.116+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8748101980 con 0x7f8748107390 2024-04-17T07:48:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.116+0000 7f8745600700 1 -- 172.21.15.46:0/844934401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8738004000 con 0x7f8748107390 2024-04-17T07:48:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.116+0000 7f8745600700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f872c0396b0 0x7f872c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.117+0000 7f8745600700 1 -- 172.21.15.46:0/844934401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f873800f050 con 0x7f8748107390 2024-04-17T07:48:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.117+0000 7f873f400700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f872c0396b0 0x7f872c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.118+0000 7f873f400700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f872c0396b0 0x7f872c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8730006fd0 tx=0x7f8730006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:07.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.122+0000 7f8745600700 1 -- 172.21.15.46:0/844934401 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8738007e20 con 0x7f8748107390 2024-04-17T07:48:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.358+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f874819f930 con 0x7f8748107390 2024-04-17T07:48:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.687+0000 7f8745600700 1 -- 172.21.15.46:0/844934401 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f873804de20 con 0x7f8748107390 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.690+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f872c0396b0 msgr2=0x7f872c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.690+0000 7f874ec00700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f872c0396b0 0x7f872c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8730006fd0 tx=0x7f8730006e40 comp rx=0 tx=0).stop 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.690+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 msgr2=0x7f874819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.690+0000 7f874ec00700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f874819e5d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87380062a0 tx=0x7f8738003ce0 comp rx=0 tx=0).stop 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.691+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 shutdown_connections 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.691+0000 7f874ec00700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f872c0396b0 0x7f872c03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.691+0000 7f874ec00700 1 --2- 172.21.15.46:0/844934401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8748107390 0x7f874819e5d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.691+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 >> 172.21.15.46:0/844934401 conn(0x7f8748076eb0 msgr2=0x7f87480fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.691+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 shutdown_connections 2024-04-17T07:48:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:07.691+0000 7f874ec00700 1 -- 172.21.15.46:0/844934401 wait complete. 2024-04-17T07:48:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-04-17T07:48:08.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:08.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:08.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:08.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:08.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:48:08.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:08.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:08.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-17T07:48:08.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: Updating smithi062:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:08.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:07 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/844934401' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-17T07:48:08.404 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-04-17T07:48:08.405 INFO:tasks.cephadm:Adding mon.b on smithi053 2024-04-17T07:48:08.405 INFO:tasks.cephadm:Adding mon.c on smithi062 2024-04-17T07:48:08.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi062:172.21.15.62=c' 2024-04-17T07:48:08.645 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:09.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:48:09.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:09.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: Updating smithi062:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:48:09.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/844934401' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-17T07:48:09.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:48:09.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:09.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:09.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.141+0000 7fbfcb730700 1 -- 172.21.15.62:0/1197339525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 msgr2=0x7fbfc41019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.141+0000 7fbfcb730700 1 --2- 172.21.15.62:0/1197339525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc41019a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009910 tx=0x7fbfb4009c20 comp rx=0 tx=0).stop 2024-04-17T07:48:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.141+0000 7fbfcb730700 1 -- 172.21.15.62:0/1197339525 shutdown_connections 2024-04-17T07:48:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.141+0000 7fbfcb730700 1 --2- 172.21.15.62:0/1197339525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc41019a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.141+0000 7fbfcb730700 1 -- 172.21.15.62:0/1197339525 >> 172.21.15.62:0/1197339525 conn(0x7fbfc40fce70 msgr2=0x7fbfc40ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.141+0000 7fbfcb730700 1 -- 172.21.15.62:0/1197339525 shutdown_connections 2024-04-17T07:48:10.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.141+0000 7fbfcb730700 1 -- 172.21.15.62:0/1197339525 wait complete. 2024-04-17T07:48:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.142+0000 7fbfcb730700 1 Processor -- start 2024-04-17T07:48:10.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.142+0000 7fbfcb730700 1 -- start start 2024-04-17T07:48:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.142+0000 7fbfcb730700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.142+0000 7fbfcb730700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc410b990 con 0x7fbfc41015c0 2024-04-17T07:48:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.143+0000 7fbfc94cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.143+0000 7fbfc94cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc4198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.62:37118/0 (socket says 172.21.15.62:37118) 2024-04-17T07:48:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.143+0000 7fbfc94cc700 1 -- 172.21.15.62:0/321676771 learned_addr learned my addr 172.21.15.62:0/321676771 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:48:10.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.143+0000 7fbfc94cc700 1 -- 172.21.15.62:0/321676771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc4199350 con 0x7fbfc41015c0 2024-04-17T07:48:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.144+0000 7fbfc94cc700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc4198e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4005850 tx=0x7fbfb4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.144+0000 7fbfba7fc700 1 -- 172.21.15.62:0/321676771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb4026070 con 0x7fbfc41015c0 2024-04-17T07:48:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.144+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc4199640 con 0x7fbfc41015c0 2024-04-17T07:48:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.144+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc4199b00 con 0x7fbfc41015c0 2024-04-17T07:48:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.144+0000 7fbfba7fc700 1 -- 172.21.15.62:0/321676771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfb400bbd0 con 0x7fbfc41015c0 2024-04-17T07:48:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.144+0000 7fbfba7fc700 1 -- 172.21.15.62:0/321676771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfb40039b0 con 0x7fbfc41015c0 2024-04-17T07:48:10.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.145+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa8005330 con 0x7fbfc41015c0 2024-04-17T07:48:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.146+0000 7fbfba7fc700 1 -- 172.21.15.62:0/321676771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbfb4029070 con 0x7fbfc41015c0 2024-04-17T07:48:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.146+0000 7fbfba7fc700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbfb0039650 0x7fbfb003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:10.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.146+0000 7fbfba7fc700 1 -- 172.21.15.62:0/321676771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbfb401e070 con 0x7fbfc41015c0 2024-04-17T07:48:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.146+0000 7fbfc8ccb700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbfb0039650 0x7fbfb003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.147+0000 7fbfc8ccb700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbfb0039650 0x7fbfb003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006fd0 tx=0x7fbfc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:10.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.161+0000 7fbfba7fc700 1 -- 172.21.15.62:0/321676771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbfb4013070 con 0x7fbfc41015c0 2024-04-17T07:48:10.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.383+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi062:172.21.15.62=c", "target": ["mon-mgr", ""]}) -- 0x7fbfa8000c70 con 0x7fbfb0039650 2024-04-17T07:48:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.399+0000 7fbfba7fc700 1 -- 172.21.15.62:0/321676771 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbfa8000c70 con 0x7fbfb0039650 2024-04-17T07:48:10.398 INFO:teuthology.orchestra.run.smithi062.stdout:Scheduled mon update... 2024-04-17T07:48:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbfb0039650 msgr2=0x7fbfb003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbfb0039650 0x7fbfb003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006fd0 tx=0x7fbfc0006e40 comp rx=0 tx=0).stop 2024-04-17T07:48:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 msgr2=0x7fbfc4198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc4198e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4005850 tx=0x7fbfb4005030 comp rx=0 tx=0).stop 2024-04-17T07:48:10.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 shutdown_connections 2024-04-17T07:48:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbfb0039650 0x7fbfb003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 --2- 172.21.15.62:0/321676771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc41015c0 0x7fbfc4198e10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 >> 172.21.15.62:0/321676771 conn(0x7fbfc40fce70 msgr2=0x7fbfc4102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 shutdown_connections 2024-04-17T07:48:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:10.401+0000 7fbfcb730700 1 -- 172.21.15.62:0/321676771 wait complete. 2024-04-17T07:48:11.096 DEBUG:teuthology.orchestra.run.smithi053:mon.b> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.b.service 2024-04-17T07:48:11.099 DEBUG:teuthology.orchestra.run.smithi062:mon.c> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.c.service 2024-04-17T07:48:11.102 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T07:48:11.102 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:11.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:12.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:11 smithi046 ceph-mon[32038]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi062:172.21.15.62=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:48:12.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:11 smithi046 ceph-mon[32038]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi062:172.21.15.62=c;count:3 2024-04-17T07:48:12.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:11 smithi046 ceph-mon[32038]: Deploying daemon mon.c on smithi062 2024-04-17T07:48:13.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:12 smithi046 ceph-mon[32038]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:14.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.818+0000 7fa728152700 1 -- 172.21.15.62:0/1302034269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 msgr2=0x7fa7201019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.818+0000 7fa728152700 1 --2- 172.21.15.62:0/1302034269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa7201019e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009910 tx=0x7fa71c009c20 comp rx=0 tx=0).stop 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.818+0000 7fa728152700 1 -- 172.21.15.62:0/1302034269 shutdown_connections 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.818+0000 7fa728152700 1 --2- 172.21.15.62:0/1302034269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa7201019e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.818+0000 7fa728152700 1 -- 172.21.15.62:0/1302034269 >> 172.21.15.62:0/1302034269 conn(0x7fa720076eb0 msgr2=0x7fa7200772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.818+0000 7fa728152700 1 -- 172.21.15.62:0/1302034269 shutdown_connections 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.818+0000 7fa728152700 1 -- 172.21.15.62:0/1302034269 wait complete. 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.819+0000 7fa728152700 1 Processor -- start 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.819+0000 7fa728152700 1 -- start start 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.819+0000 7fa728152700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa720198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:14.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.819+0000 7fa728152700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa72010b980 con 0x7fa720101600 2024-04-17T07:48:14.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.820+0000 7fa725eee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa720198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.820+0000 7fa725eee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa720198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.62:37134/0 (socket says 172.21.15.62:37134) 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.820+0000 7fa725eee700 1 -- 172.21.15.62:0/1676468212 learned_addr learned my addr 172.21.15.62:0/1676468212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.820+0000 7fa725eee700 1 -- 172.21.15.62:0/1676468212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa720199370 con 0x7fa720101600 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.821+0000 7fa725eee700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa720198e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa71c005850 tx=0x7fa71c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.821+0000 7fa716ffd700 1 -- 172.21.15.62:0/1676468212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa71c026070 con 0x7fa720101600 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.821+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa720199660 con 0x7fa720101600 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.821+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa720199b20 con 0x7fa720101600 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.821+0000 7fa716ffd700 1 -- 172.21.15.62:0/1676468212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71c00bbd0 con 0x7fa720101600 2024-04-17T07:48:14.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.821+0000 7fa716ffd700 1 -- 172.21.15.62:0/1676468212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa71c0039b0 con 0x7fa720101600 2024-04-17T07:48:14.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.823+0000 7fa716ffd700 1 -- 172.21.15.62:0/1676468212 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa71c029070 con 0x7fa720101600 2024-04-17T07:48:14.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.823+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa720102a90 con 0x7fa720101600 2024-04-17T07:48:14.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.823+0000 7fa716ffd700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa70c0396b0 0x7fa70c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:14.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.823+0000 7fa716ffd700 1 -- 172.21.15.62:0/1676468212 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa71c01e070 con 0x7fa720101600 2024-04-17T07:48:14.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.823+0000 7fa7256ed700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa70c0396b0 0x7fa70c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:14.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.825+0000 7fa7256ed700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa70c0396b0 0x7fa70c03bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa710006fd0 tx=0x7fa710006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:14.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:14.839+0000 7fa716ffd700 1 -- 172.21.15.62:0/1676468212 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa71c013070 con 0x7fa720101600 2024-04-17T07:48:15.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:14 smithi046 ceph-mon[32038]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:15.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.140+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa7200640b0 con 0x7fa720101600 2024-04-17T07:48:15.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.141+0000 7fa716ffd700 1 -- 172.21.15.62:0/1676468212 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fa71c027260 con 0x7fa720101600 2024-04-17T07:48:15.141 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:48:15.141 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","modified":"2024-04-17T07:46:07.693966Z","created":"2024-04-17T07:46:07.693966Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-17T07:48:15.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa70c0396b0 msgr2=0x7fa70c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa70c0396b0 0x7fa70c03bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa710006fd0 tx=0x7fa710006e40 comp rx=0 tx=0).stop 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 msgr2=0x7fa720198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa720198e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa71c005850 tx=0x7fa71c005030 comp rx=0 tx=0).stop 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 shutdown_connections 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa70c0396b0 0x7fa70c03bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 --2- 172.21.15.62:0/1676468212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720101600 0x7fa720198e30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 >> 172.21.15.62:0/1676468212 conn(0x7fa720076eb0 msgr2=0x7fa7201043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 shutdown_connections 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:15.143+0000 7fa728152700 1 -- 172.21.15.62:0/1676468212 wait complete. 2024-04-17T07:48:15.143 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-04-17T07:48:16.931 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T07:48:16.931 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T07:48:17.190 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: mon.a calling monitor election 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: mon.c calling monitor election 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:20.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: monmap epoch 2 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: last_changed 2024-04-17T07:48:15.218018+0000 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: min_mon_release 19 (squid) 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: election_strategy: 1 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: 1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.c 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: fsmap 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: mgrmap e13: x(active, since 58s) 2024-04-17T07:48:20.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: overall HEALTH_OK 2024-04-17T07:48:20.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:20.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:21.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.536+0000 7f4598b0a700 1 -- 172.21.15.62:0/4055016718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 msgr2=0x7f457c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:21.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.536+0000 7f4598b0a700 1 --2- 172.21.15.62:0/4055016718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 0x7f457c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4594101390 tx=0x7f458400feb0 comp rx=0 tx=0).stop 2024-04-17T07:48:21.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.536+0000 7f4598b0a700 1 -- 172.21.15.62:0/4055016718 shutdown_connections 2024-04-17T07:48:21.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.536+0000 7f4598b0a700 1 --2- 172.21.15.62:0/4055016718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 0x7f457c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:21.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.536+0000 7f4598b0a700 1 -- 172.21.15.62:0/4055016718 >> 172.21.15.62:0/4055016718 conn(0x7f45940fcf50 msgr2=0x7f45940ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:21.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.536+0000 7f4598b0a700 1 -- 172.21.15.62:0/4055016718 shutdown_connections 2024-04-17T07:48:21.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.536+0000 7f4598b0a700 1 -- 172.21.15.62:0/4055016718 wait complete. 2024-04-17T07:48:21.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.537+0000 7f4598b0a700 1 Processor -- start 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.537+0000 7f4598b0a700 1 -- start start 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4598b0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 0x7f459419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4598b0a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 0x7f459419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4598b0a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4594109500 con 0x7f457c0035c0 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4598b0a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4594109380 con 0x7f459419a820 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4591d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 0x7f459419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4591d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 0x7f459419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49008/0 (socket says 172.21.15.62:49008) 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4591d9b700 1 -- 172.21.15.62:0/904947340 learned_addr learned my addr 172.21.15.62:0/904947340 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4591d9b700 1 -- 172.21.15.62:0/904947340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 msgr2=0x7f459419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4591d9b700 1 -- 172.21.15.62:0/904947340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 msgr2=0x7f459419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4591d9b700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 0x7f459419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T07:48:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f4591d9b700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 0x7f459419ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:48:21.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.538+0000 7f459259c700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 0x7f459419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:21.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.539+0000 7f459259c700 1 -- 172.21.15.62:0/904947340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 msgr2=0x7f459419ebb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:48:21.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.539+0000 7f459259c700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 0x7f459419ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:21.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.539+0000 7f459259c700 1 -- 172.21.15.62:0/904947340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459419ae60 con 0x7f457c0035c0 2024-04-17T07:48:21.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.540+0000 7f459259c700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 0x7f459419a2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4584010010 tx=0x7f45840046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.540+0000 7f45837fe700 1 -- 172.21.15.62:0/904947340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4584013b00 con 0x7f457c0035c0 2024-04-17T07:48:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.540+0000 7f45837fe700 1 -- 172.21.15.62:0/904947340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4584004510 con 0x7f457c0035c0 2024-04-17T07:48:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.540+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459419f0f0 con 0x7f457c0035c0 2024-04-17T07:48:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.540+0000 7f45837fe700 1 -- 172.21.15.62:0/904947340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4584003d80 con 0x7f457c0035c0 2024-04-17T07:48:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.540+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459419f650 con 0x7f457c0035c0 2024-04-17T07:48:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.542+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459419f9c0 con 0x7f457c0035c0 2024-04-17T07:48:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.542+0000 7f45837fe700 1 -- 172.21.15.62:0/904947340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4584023070 con 0x7f457c0035c0 2024-04-17T07:48:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.542+0000 7f45837fe700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4574039440 0x7f457403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.543+0000 7f45837fe700 1 -- 172.21.15.62:0/904947340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4584020050 con 0x7f457c0035c0 2024-04-17T07:48:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.543+0000 7f4591d9b700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4574039440 0x7f457403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.544+0000 7f4591d9b700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4574039440 0x7f457403b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4588009180 tx=0x7f4588008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:21.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.558+0000 7f45837fe700 1 -- 172.21.15.62:0/904947340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4584028020 con 0x7f457c0035c0 2024-04-17T07:48:21.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:21.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:21.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:21.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:21 smithi046 ceph-mon[32038]: Deploying daemon mon.b on smithi053 2024-04-17T07:48:21.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:21.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.855+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f459419b640 con 0x7f457c0035c0 2024-04-17T07:48:21.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.856+0000 7f45837fe700 1 -- 172.21.15.62:0/904947340 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f4584011e60 con 0x7f457c0035c0 2024-04-17T07:48:21.857 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:48:21.857 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","modified":"2024-04-17T07:48:15.218018Z","created":"2024-04-17T07:46:07.693966Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4574039440 msgr2=0x7f457403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4574039440 0x7f457403b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4588009180 tx=0x7f4588008040 comp rx=0 tx=0).stop 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 msgr2=0x7f459419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 0x7f459419a2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4584010010 tx=0x7f45840046f0 comp rx=0 tx=0).stop 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 shutdown_connections 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4574039440 0x7f457403b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0035c0 0x7f459419a2e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 --2- 172.21.15.62:0/904947340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f459419a820 0x7f459419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 >> 172.21.15.62:0/904947340 conn(0x7f45940fcf50 msgr2=0x7f459406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 shutdown_connections 2024-04-17T07:48:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:21.858+0000 7f4598b0a700 1 -- 172.21.15.62:0/904947340 wait complete. 2024-04-17T07:48:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 2 2024-04-17T07:48:22.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:22 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/904947340' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:48:22.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:22 smithi046 ceph-mon[32038]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:22.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:22 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/904947340' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:48:23.453 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T07:48:23.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T07:48:23.710 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:48:29.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: mon.c calling monitor election 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: mon.a calling monitor election 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: monmap epoch 3 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: last_changed 2024-04-17T07:48:24.216109+0000 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: min_mon_release 19 (squid) 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: election_strategy: 1 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: 1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.c 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: fsmap 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: mgrmap e13: x(active, since 67s) 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: overall HEALTH_OK 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:29.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:29.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:29.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.568+0000 7fd74bcd5700 1 -- 172.21.15.62:0/2007617477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7300035c0 msgr2=0x7fd730005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:29.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.568+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/2007617477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7300035c0 0x7fd730005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd744102470 tx=0x7fd73800fe60 comp rx=0 tx=0).stop 2024-04-17T07:48:29.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.568+0000 7fd74bcd5700 1 -- 172.21.15.62:0/2007617477 shutdown_connections 2024-04-17T07:48:29.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.568+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/2007617477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7300035c0 0x7fd730005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:29.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.568+0000 7fd74bcd5700 1 -- 172.21.15.62:0/2007617477 >> 172.21.15.62:0/2007617477 conn(0x7fd7440fe050 msgr2=0x7fd744100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:29.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.568+0000 7fd74bcd5700 1 -- 172.21.15.62:0/2007617477 shutdown_connections 2024-04-17T07:48:29.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.568+0000 7fd74bcd5700 1 -- 172.21.15.62:0/2007617477 wait complete. 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.569+0000 7fd74bcd5700 1 Processor -- start 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.569+0000 7fd74bcd5700 1 -- start start 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd74bcd5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 0x7fd7441981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd74bcd5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 0x7fd74419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd74bcd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74419cfc0 0x7fd74419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd74bcd5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410a590 con 0x7fd74419cfc0 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd74bcd5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410a410 con 0x7fd7300035c0 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd74bcd5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410a710 con 0x7fd7441986f0 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749270700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 0x7fd74419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:29.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749270700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 0x7fd74419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48218/0 (socket says 172.21.15.62:48218) 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749270700 1 -- 172.21.15.62:0/4288334779 learned_addr learned my addr 172.21.15.62:0/4288334779 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749a71700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 0x7fd7441981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749270700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 msgr2=0x7fd74419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749270700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 msgr2=0x7fd74419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749270700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 0x7fd74419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749270700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 0x7fd74419ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749a71700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 msgr2=0x7fd7441981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749a71700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 msgr2=0x7fd7441981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749a71700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 0x7fd7441981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd74a272700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74419cfc0 0x7fd74419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:29.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.570+0000 7fd749a71700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 0x7fd7441981b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:48:29.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.571+0000 7fd74a272700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 msgr2=0x7fd74419ca80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:48:29.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.571+0000 7fd74a272700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 0x7fd74419ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:29.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.571+0000 7fd74a272700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 msgr2=0x7fd7441981b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.571+0000 7fd74a272700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 0x7fd7441981b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.571+0000 7fd74a272700 1 -- 172.21.15.62:0/4288334779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441a4d30 con 0x7fd74419cfc0 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.571+0000 7fd74a272700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74419cfc0 0x7fd74419d470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00bd50 tx=0x7fd73c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.572+0000 7fd736ffd700 1 -- 172.21.15.62:0/4288334779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c019070 con 0x7fd74419cfc0 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.572+0000 7fd736ffd700 1 -- 172.21.15.62:0/4288334779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73c00cd10 con 0x7fd74419cfc0 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.572+0000 7fd736ffd700 1 -- 172.21.15.62:0/4288334779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0127e0 con 0x7fd74419cfc0 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.572+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7441a4f60 con 0x7fd74419cfc0 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.572+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441a5470 con 0x7fd74419cfc0 2024-04-17T07:48:29.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.573+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7440640b0 con 0x7fd74419cfc0 2024-04-17T07:48:29.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.574+0000 7fd736ffd700 1 -- 172.21.15.62:0/4288334779 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd73c01c070 con 0x7fd74419cfc0 2024-04-17T07:48:29.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.574+0000 7fd736ffd700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd728039ae0 0x7fd72803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:29.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.575+0000 7fd736ffd700 1 -- 172.21.15.62:0/4288334779 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd73c04c6d0 con 0x7fd74419cfc0 2024-04-17T07:48:29.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.575+0000 7fd749a71700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd728039ae0 0x7fd72803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:29.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.577+0000 7fd749a71700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd728039ae0 0x7fd72803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd738020040 tx=0x7fd738001e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.591+0000 7fd736ffd700 1 -- 172.21.15.62:0/4288334779 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd73c019210 con 0x7fd74419cfc0 2024-04-17T07:48:29.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: mon.c calling monitor election 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: mon.a calling monitor election 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: monmap epoch 3 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: last_changed 2024-04-17T07:48:24.216109+0000 2024-04-17T07:48:29.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: min_mon_release 19 (squid) 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: election_strategy: 1 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: 1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.c 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: fsmap 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: mgrmap e13: x(active, since 67s) 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: overall HEALTH_OK 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:29.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:29.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.890+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd7441a5790 con 0x7fd74419cfc0 2024-04-17T07:48:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.890+0000 7fd736ffd700 1 -- 172.21.15.62:0/4288334779 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fd73c04b0e0 con 0x7fd74419cfc0 2024-04-17T07:48:29.891 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-17T07:48:29.891 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","modified":"2024-04-17T07:48:24.216109Z","created":"2024-04-17T07:46:07.693966Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd728039ae0 msgr2=0x7fd72803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd728039ae0 0x7fd72803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd738020040 tx=0x7fd738001e70 comp rx=0 tx=0).stop 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74419cfc0 msgr2=0x7fd74419d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74419cfc0 0x7fd74419d470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00bd50 tx=0x7fd73c00be30 comp rx=0 tx=0).stop 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 shutdown_connections 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd728039ae0 0x7fd72803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd74419cfc0 0x7fd74419d470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7300035c0 0x7fd7441981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 --2- 172.21.15.62:0/4288334779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441986f0 0x7fd74419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 >> 172.21.15.62:0/4288334779 conn(0x7fd7440fe050 msgr2=0x7fd7440ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 shutdown_connections 2024-04-17T07:48:29.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:29.894+0000 7fd74bcd5700 1 -- 172.21.15.62:0/4288334779 wait complete. 2024-04-17T07:48:29.896 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 3 2024-04-17T07:48:30.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:30.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:30 smithi062 ceph-mon[36507]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:30.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:30.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:30 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/4288334779' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:48:30.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:30.499 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-17T07:48:30.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph config generate-minimal-conf 2024-04-17T07:48:30.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:30.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:30 smithi046 ceph-mon[32038]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:30.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:30.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:30 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/4288334779' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:48:30.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:30.946 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:48:31.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: mon.b calling monitor election 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: mon.b calling monitor election 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: mon.c calling monitor election 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: mon.a calling monitor election 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: monmap epoch 3 2024-04-17T07:48:32.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: last_changed 2024-04-17T07:48:24.216109+0000 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: min_mon_release 19 (squid) 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: election_strategy: 1 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: 1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.c 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: fsmap 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: mgrmap e13: x(active, since 69s) 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: overall HEALTH_OK 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-17T07:48:32.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: Updating smithi062:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: Updating smithi053:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: Updating smithi046:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:32.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: mon.b calling monitor election 2024-04-17T07:48:32.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: mon.b calling monitor election 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: mon.c calling monitor election 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: mon.a calling monitor election 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: monmap epoch 3 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: last_changed 2024-04-17T07:48:24.216109+0000 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: min_mon_release 19 (squid) 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: election_strategy: 1 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:48:32.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: 1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.c 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: fsmap 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: mgrmap e13: x(active, since 69s) 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: overall HEALTH_OK 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: Updating smithi062:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: Updating smithi053:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: Updating smithi046:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:32.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: mon.b calling monitor election 2024-04-17T07:48:32.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: mon.b calling monitor election 2024-04-17T07:48:32.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: mon.c calling monitor election 2024-04-17T07:48:32.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: mon.a calling monitor election 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: monmap epoch 3 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: last_changed 2024-04-17T07:48:24.216109+0000 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: created 2024-04-17T07:46:07.693966+0000 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: min_mon_release 19 (squid) 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: election_strategy: 1 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: 1: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.c 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: fsmap 2024-04-17T07:48:32.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: mgrmap e13: x(active, since 69s) 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: overall HEALTH_OK 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: Updating smithi053:/etc/ceph/ceph.conf 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: Updating smithi062:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: Updating smithi053:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:32.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: Updating smithi046:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:48:32.725 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:48:33.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.166+0000 7feb74c00700 1 -- 172.21.15.46:0/3819261977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb701084f0 msgr2=0x7feb701088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:33.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.166+0000 7feb74c00700 1 --2- 172.21.15.46:0/3819261977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb701084f0 0x7feb701088d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feb60009910 tx=0x7feb60009c20 comp rx=0 tx=0).stop 2024-04-17T07:48:33.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.167+0000 7feb74c00700 1 -- 172.21.15.46:0/3819261977 shutdown_connections 2024-04-17T07:48:33.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.167+0000 7feb74c00700 1 --2- 172.21.15.46:0/3819261977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb701084f0 0x7feb701088d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.167+0000 7feb74c00700 1 -- 172.21.15.46:0/3819261977 >> 172.21.15.46:0/3819261977 conn(0x7feb70076eb0 msgr2=0x7feb700772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:33.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.167+0000 7feb74c00700 1 -- 172.21.15.46:0/3819261977 shutdown_connections 2024-04-17T07:48:33.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.167+0000 7feb74c00700 1 -- 172.21.15.46:0/3819261977 wait complete. 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.168+0000 7feb74c00700 1 Processor -- start 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.168+0000 7feb74c00700 1 -- start start 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.168+0000 7feb74c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb701084f0 0x7feb7019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.168+0000 7feb74c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb7019a9a0 0x7feb7019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.168+0000 7feb74c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7019f1d0 0x7feb7019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb74c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb7010ca80 con 0x7feb7019a9a0 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb74c00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb7010c900 con 0x7feb7019f1d0 2024-04-17T07:48:33.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb74c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb7010cc00 con 0x7feb701084f0 2024-04-17T07:48:33.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb6d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb7019a9a0 0x7feb7019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:33.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb6d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb7019a9a0 0x7feb7019ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41248/0 (socket says 172.21.15.46:41248) 2024-04-17T07:48:33.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb6d400700 1 -- 172.21.15.46:0/3031191679 learned_addr learned my addr 172.21.15.46:0/3031191679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:48:33.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb6de00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb701084f0 0x7feb7019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:33.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.169+0000 7feb6e800700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7019f1d0 0x7feb7019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:33.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb6d400700 1 -- 172.21.15.46:0/3031191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb701084f0 msgr2=0x7feb7019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:33.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb6d400700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb701084f0 0x7feb7019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb6d400700 1 -- 172.21.15.46:0/3031191679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7019f1d0 msgr2=0x7feb7019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:33.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb6d400700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7019f1d0 0x7feb7019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb6d400700 1 -- 172.21.15.46:0/3031191679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb701a6f40 con 0x7feb7019a9a0 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb6d400700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb7019a9a0 0x7feb7019ec90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00b780 tx=0x7feb5c00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb66a00700 1 -- 172.21.15.46:0/3031191679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c016070 con 0x7feb7019a9a0 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb66a00700 1 -- 172.21.15.46:0/3031191679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb5c0036a0 con 0x7feb7019a9a0 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb701a7170 con 0x7feb7019a9a0 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.170+0000 7feb66a00700 1 -- 172.21.15.46:0/3031191679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c011af0 con 0x7feb7019a9a0 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.171+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701a76b0 con 0x7feb7019a9a0 2024-04-17T07:48:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.172+0000 7feb66a00700 1 -- 172.21.15.46:0/3031191679 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feb5c019070 con 0x7feb7019a9a0 2024-04-17T07:48:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.172+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb700640b0 con 0x7feb7019a9a0 2024-04-17T07:48:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.172+0000 7feb66a00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb54039ae0 0x7feb5403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.172+0000 7feb66a00700 1 -- 172.21.15.46:0/3031191679 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feb5c04b490 con 0x7feb7019a9a0 2024-04-17T07:48:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.173+0000 7feb6de00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb54039ae0 0x7feb5403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:33.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.173+0000 7feb6de00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb54039ae0 0x7feb5403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feb60006010 tx=0x7feb600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:33.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.178+0000 7feb66a00700 1 -- 172.21.15.46:0/3031191679 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7feb5c019bb0 con 0x7feb7019a9a0 2024-04-17T07:48:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.406+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7feb701a79a0 con 0x7feb7019a9a0 2024-04-17T07:48:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.407+0000 7feb66a00700 1 -- 172.21.15.46:0/3031191679 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7feb5c01a920 con 0x7feb7019a9a0 2024-04-17T07:48:33.407 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:33.407 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-17T07:48:33.407 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:48:33.407 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] 2024-04-17T07:48:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.410+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb54039ae0 msgr2=0x7feb5403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.410+0000 7feb74c00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb54039ae0 0x7feb5403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feb60006010 tx=0x7feb600058e0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.410+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb7019a9a0 msgr2=0x7feb7019ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.410+0000 7feb74c00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb7019a9a0 0x7feb7019ec90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00b780 tx=0x7feb5c00ba90 comp rx=0 tx=0).stop 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 shutdown_connections 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb54039ae0 0x7feb5403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7019f1d0 0x7feb7019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb701084f0 0x7feb7019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 --2- 172.21.15.46:0/3031191679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb7019a9a0 0x7feb7019ec90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 >> 172.21.15.46:0/3031191679 conn(0x7feb70076eb0 msgr2=0x7feb700fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 shutdown_connections 2024-04-17T07:48:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:33.411+0000 7feb74c00700 1 -- 172.21.15.46:0/3031191679 wait complete. 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:33.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:33.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:33.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: Reconfiguring daemon mon.a on smithi046 2024-04-17T07:48:33.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:33 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3031191679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:34.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: Reconfiguring daemon mon.a on smithi046 2024-04-17T07:48:34.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:33 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3031191679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:34.089 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: Reconfiguring daemon mon.a on smithi046 2024-04-17T07:48:34.090 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:33 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3031191679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:34.151 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-17T07:48:34.151 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:48:34.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T07:48:34.254 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:48:34.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:48:34.401 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:48:34.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T07:48:34.440 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:48:34.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:48:34.513 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:48:34.513 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T07:48:34.547 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:48:34.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:48:34.618 INFO:tasks.cephadm:Adding mgr.x on smithi046 2024-04-17T07:48:34.618 INFO:tasks.cephadm:Adding mgr.y on smithi053 2024-04-17T07:48:34.618 INFO:tasks.cephadm:Adding mgr.z on smithi062 2024-04-17T07:48:34.619 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch apply mgr '3;smithi046=x;smithi053=y;smithi062=z' 2024-04-17T07:48:34.905 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:48:35.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:34 smithi053 ceph-mon[36437]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:35.269 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:34 smithi062 ceph-mon[36507]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:35.302 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:34 smithi046 ceph-mon[32038]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:36.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.440+0000 7f3149630700 1 -- 172.21.15.62:0/658616322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3144100e30 msgr2=0x7f3144101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.440+0000 7f3149630700 1 --2- 172.21.15.62:0/658616322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3144100e30 0x7f3144101250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f313c00b0a0 tx=0x7f313c00b3b0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.440+0000 7f3149630700 1 -- 172.21.15.62:0/658616322 shutdown_connections 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.440+0000 7f3149630700 1 --2- 172.21.15.62:0/658616322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3144100e30 0x7f3144101250 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.440+0000 7f3149630700 1 -- 172.21.15.62:0/658616322 >> 172.21.15.62:0/658616322 conn(0x7f31440fca50 msgr2=0x7f31440fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.440+0000 7f3149630700 1 -- 172.21.15.62:0/658616322 shutdown_connections 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.440+0000 7f3149630700 1 -- 172.21.15.62:0/658616322 wait complete. 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 Processor -- start 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 -- start start 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3144100e30 0x7f3144199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f314419a500 0x7f314419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314419a980 0x7f31441a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f314410a7e0 con 0x7f314419a980 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f314410a660 con 0x7f3144100e30 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3149630700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f314410a960 con 0x7f314419a500 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f31427fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f314419a500 0x7f314419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f31427fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f314419a500 0x7f314419f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48224/0 (socket says 172.21.15.62:48224) 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f31427fc700 1 -- 172.21.15.62:0/4125105172 learned_addr learned my addr 172.21.15.62:0/4125105172 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.441+0000 7f3142ffd700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3144100e30 0x7f3144199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f31427fc700 1 -- 172.21.15.62:0/4125105172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3144100e30 msgr2=0x7f3144199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f31437fe700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314419a980 0x7f31441a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f31427fc700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3144100e30 0x7f3144199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f31427fc700 1 -- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314419a980 msgr2=0x7f31441a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f31427fc700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314419a980 0x7f31441a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f31427fc700 1 -- 172.21.15.62:0/4125105172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31441a23d0 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f3142ffd700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3144100e30 0x7f3144199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f31427fc700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f314419a500 0x7f314419f570 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f312c00b780 tx=0x7f312c00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f313bfff700 1 -- 172.21.15.62:0/4125105172 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c004230 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f313bfff700 1 -- 172.21.15.62:0/4125105172 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f312c0043d0 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f313bfff700 1 -- 172.21.15.62:0/4125105172 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c011c90 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31441a26c0 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.442+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31441a2c80 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.443+0000 7f313bfff700 1 -- 172.21.15.62:0/4125105172 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f312c01a400 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.443+0000 7f313bfff700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3130039ae0 0x7f313003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.443+0000 7f313bfff700 1 -- 172.21.15.62:0/4125105172 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f312c04d500 con 0x7f314419a500 2024-04-17T07:48:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.443+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314402df90 con 0x7f314419a500 2024-04-17T07:48:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.444+0000 7f3142ffd700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3130039ae0 0x7f313003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.445+0000 7f3142ffd700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3130039ae0 0x7f313003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f313c0076a0 tx=0x7f313c005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.449+0000 7f313bfff700 1 -- 172.21.15.62:0/4125105172 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f312c020500 con 0x7f314419a500 2024-04-17T07:48:36.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:36.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:36.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:36.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: Reconfiguring mon.b (monmap changed)... 2024-04-17T07:48:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:36 smithi062 ceph-mon[36507]: Reconfiguring daemon mon.b on smithi053 2024-04-17T07:48:36.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.674+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi062=z", "target": ["mon-mgr", ""]}) -- 0x7f3144062e90 con 0x7f3130039ae0 2024-04-17T07:48:36.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.688+0000 7f313bfff700 1 -- 172.21.15.62:0/4125105172 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3144062e90 con 0x7f3130039ae0 2024-04-17T07:48:36.688 INFO:teuthology.orchestra.run.smithi062.stdout:Scheduled mgr update... 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3130039ae0 msgr2=0x7f313003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3130039ae0 0x7f313003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f313c0076a0 tx=0x7f313c005040 comp rx=0 tx=0).stop 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f314419a500 msgr2=0x7f314419f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f314419a500 0x7f314419f570 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f312c00b780 tx=0x7f312c00ba90 comp rx=0 tx=0).stop 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 shutdown_connections 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3130039ae0 0x7f313003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314419a980 0x7f31441a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3144100e30 0x7f3144199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 --2- 172.21.15.62:0/4125105172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f314419a500 0x7f314419f570 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 >> 172.21.15.62:0/4125105172 conn(0x7f31440fca50 msgr2=0x7f31440fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 shutdown_connections 2024-04-17T07:48:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:48:36.690+0000 7f3149630700 1 -- 172.21.15.62:0/4125105172 wait complete. 2024-04-17T07:48:36.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:36.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:36.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:36.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: Reconfiguring mon.b (monmap changed)... 2024-04-17T07:48:36.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:36.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:36.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:36.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:36 smithi053 ceph-mon[36437]: Reconfiguring daemon mon.b on smithi053 2024-04-17T07:48:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: Reconfiguring mon.b (monmap changed)... 2024-04-17T07:48:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:36.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:36.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:36 smithi046 ceph-mon[32038]: Reconfiguring daemon mon.b on smithi053 2024-04-17T07:48:37.468 DEBUG:teuthology.orchestra.run.smithi053:mgr.y> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.y.service 2024-04-17T07:48:37.471 DEBUG:teuthology.orchestra.run.smithi062:mgr.z> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.z.service 2024-04-17T07:48:37.473 INFO:tasks.cephadm:Deploying OSDs... 2024-04-17T07:48:37.473 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:48:37.473 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T07:48:37.534 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T07:48:37.534 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-04-17T07:48:37.593 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 767 Links: 1 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-17 07:47:38.228254880 +0000 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-17 07:42:08.567816703 +0000 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-17 07:42:08.567816703 +0000 2024-04-17T07:48:37.594 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-17 07:42:08.567816703 +0000 2024-04-17T07:48:37.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi062=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: Saving service mgr spec with placement smithi046=x;smithi053=y;smithi062=z;count:3 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: Reconfiguring mon.c (monmap changed)... 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:37.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:37.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:37.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:37 smithi062 ceph-mon[36507]: Reconfiguring daemon mon.c on smithi062 2024-04-17T07:48:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-17T07:48:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-17T07:48:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000247896 s, 2.1 MB/s 2024-04-17T07:48:37.701 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T07:48:37.760 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-04-17T07:48:37.813 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T07:48:37.813 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:37.813 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 789 Links: 1 2024-04-17T07:48:37.814 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:37.814 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:37.814 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-17 07:47:38.339253709 +0000 2024-04-17T07:48:37.814 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-17 07:42:09.198810085 +0000 2024-04-17T07:48:37.814 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-17 07:42:09.198810085 +0000 2024-04-17T07:48:37.814 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-17 07:42:09.198810085 +0000 2024-04-17T07:48:37.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T07:48:37.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-17T07:48:37.907 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-17T07:48:37.907 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000277133 s, 1.8 MB/s 2024-04-17T07:48:37.911 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T07:48:37.966 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-04-17T07:48:37.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi062=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:48:37.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: Saving service mgr spec with placement smithi046=x;smithi053=y;smithi062=z;count:3 2024-04-17T07:48:37.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:37.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:37.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:37.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: Reconfiguring mon.c (monmap changed)... 2024-04-17T07:48:37.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:37.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:37.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:37.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:37 smithi053 ceph-mon[36437]: Reconfiguring daemon mon.c on smithi062 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 805 Links: 1 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-17 07:47:38.418252876 +0000 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-17 07:42:09.798803792 +0000 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-17 07:42:09.798803792 +0000 2024-04-17T07:48:38.015 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-17 07:42:09.798803792 +0000 2024-04-17T07:48:38.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T07:48:38.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi053=y;smithi062=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:48:38.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: Saving service mgr spec with placement smithi046=x;smithi053=y;smithi062=z;count:3 2024-04-17T07:48:38.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:38.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:38.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:38.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: Reconfiguring mon.c (monmap changed)... 2024-04-17T07:48:38.101 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:48:38.102 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:48:38.102 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:38.102 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:37 smithi046 ceph-mon[32038]: Reconfiguring daemon mon.c on smithi062 2024-04-17T07:48:38.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-17T07:48:38.111 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-17T07:48:38.111 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000249864 s, 2.0 MB/s 2024-04-17T07:48:38.116 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T07:48:38.174 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-04-17T07:48:38.226 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T07:48:38.226 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:38.226 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 826 Links: 1 2024-04-17T07:48:38.226 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:38.227 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:38.227 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-17 07:47:38.504251969 +0000 2024-04-17T07:48:38.227 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-17 07:42:10.413797342 +0000 2024-04-17T07:48:38.227 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-17 07:42:10.413797342 +0000 2024-04-17T07:48:38.227 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-17 07:42:10.413797342 +0000 2024-04-17T07:48:38.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T07:48:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-17T07:48:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-17T07:48:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000257718 s, 2.0 MB/s 2024-04-17T07:48:38.326 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T07:48:38.388 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-17T07:48:38.388 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T07:48:38.407 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T07:48:38.408 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-04-17T07:48:38.465 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T07:48:38.465 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:38.466 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-17T07:48:38.466 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:38.466 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:38.466 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-17 07:48:23.663208200 +0000 2024-04-17T07:48:38.466 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-17 07:40:33.821020513 +0000 2024-04-17T07:48:38.466 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-17 07:40:33.821020513 +0000 2024-04-17T07:48:38.466 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-17 07:40:33.821020513 +0000 2024-04-17T07:48:38.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T07:48:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-17T07:48:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-17T07:48:38.540 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000114971 s, 4.5 MB/s 2024-04-17T07:48:38.541 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T07:48:38.600 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-17 07:48:23.663208200 +0000 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-17 07:40:34.161013393 +0000 2024-04-17T07:48:38.657 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-17 07:40:34.161013393 +0000 2024-04-17T07:48:38.658 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-17 07:40:34.161013393 +0000 2024-04-17T07:48:38.658 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T07:48:38.729 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-17T07:48:38.729 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-17T07:48:38.729 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000118208 s, 4.3 MB/s 2024-04-17T07:48:38.730 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T07:48:38.788 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-04-17T07:48:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-17 07:48:23.664208179 +0000 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-17 07:40:34.499006315 +0000 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-17 07:40:34.499006315 +0000 2024-04-17T07:48:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-17 07:40:34.499006315 +0000 2024-04-17T07:48:38.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T07:48:38.918 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-17T07:48:38.918 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-17T07:48:38.918 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.00011785 s, 4.3 MB/s 2024-04-17T07:48:38.919 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T07:48:38.977 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-17 07:48:23.664208179 +0000 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-17 07:40:34.840999153 +0000 2024-04-17T07:48:39.034 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-17 07:40:34.840999153 +0000 2024-04-17T07:48:39.035 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-04-17 07:40:34.840999153 +0000 2024-04-17T07:48:39.035 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T07:48:39.105 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-17T07:48:39.105 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-17T07:48:39.105 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.666e-05 s, 5.3 MB/s 2024-04-17T07:48:39.106 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T07:48:39.163 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-17T07:48:39.163 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T07:48:39.179 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T07:48:39.179 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-04-17T07:48:39.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:39.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-17 07:48:14.039062834 +0000 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-17 07:40:35.143135784 +0000 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-17 07:40:35.143135784 +0000 2024-04-17T07:48:39.235 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-17 07:40:35.143135784 +0000 2024-04-17T07:48:39.236 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T07:48:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:39.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:48:39.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:39.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:39.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-17T07:48:39.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-17T07:48:39.306 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000271833 s, 1.9 MB/s 2024-04-17T07:48:39.307 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T07:48:39.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:39.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:39.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:39.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:39.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:39.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:39.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:48:39.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:39.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:39.363 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-04-17T07:48:39.418 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T07:48:39.418 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:39.418 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-17T07:48:39.418 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:39.418 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:39.419 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-17 07:48:14.040062819 +0000 2024-04-17T07:48:39.419 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-17 07:40:35.491130422 +0000 2024-04-17T07:48:39.419 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-17 07:40:35.491130422 +0000 2024-04-17T07:48:39.419 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-17 07:40:35.491130422 +0000 2024-04-17T07:48:39.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T07:48:39.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-17T07:48:39.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-17T07:48:39.489 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000213827 s, 2.4 MB/s 2024-04-17T07:48:39.490 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T07:48:39.546 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-17 07:48:14.040062819 +0000 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-17 07:40:35.835125121 +0000 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-17 07:40:35.835125121 +0000 2024-04-17T07:48:39.602 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-17 07:40:35.835125121 +0000 2024-04-17T07:48:39.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T07:48:39.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-17T07:48:39.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-17T07:48:39.672 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000203976 s, 2.5 MB/s 2024-04-17T07:48:39.673 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T07:48:39.729 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-04-17T07:48:39.784 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T07:48:39.784 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:48:39.784 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-17T07:48:39.784 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:48:39.784 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:48:39.785 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-17 07:48:14.041062804 +0000 2024-04-17T07:48:39.785 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-17 07:40:36.215119264 +0000 2024-04-17T07:48:39.785 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-17 07:40:36.215119264 +0000 2024-04-17T07:48:39.785 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-17 07:40:36.215119264 +0000 2024-04-17T07:48:39.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T07:48:39.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-17T07:48:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-17T07:48:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000201119 s, 2.5 MB/s 2024-04-17T07:48:39.855 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T07:48:39.911 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-04-17T07:48:39.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T07:48:40.243 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:40 smithi062 ceph-mon[36507]: Deploying daemon mgr.z on smithi062 2024-04-17T07:48:40.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:40 smithi046 ceph-mon[32038]: Deploying daemon mgr.z on smithi062 2024-04-17T07:48:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:48:40.475 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:40 smithi053 ceph-mon[36437]: Deploying daemon mgr.z on smithi062 2024-04-17T07:48:40.803 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:40 smithi062 systemd[1]: Starting Ceph mgr.z for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:48:41.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:41 smithi046 ceph-mon[32038]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:41.214 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:41 smithi062 ceph-mon[36507]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:41.214 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:40 smithi062 podman[38368]: 2024-04-17 07:48:40.830362588 +0000 UTC m=+0.018314241 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:48:41.214 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:40 smithi062 podman[38368]: 2024-04-17 07:48:40.929357446 +0000 UTC m=+0.117309090 container create 35a0481921de92e8361416940b0ae971cbb428422a32d56cd270c3b829720fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:48:41.214 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:41 smithi062 podman[38368]: 2024-04-17 07:48:41.140284902 +0000 UTC m=+0.328236552 container init 35a0481921de92e8361416940b0ae971cbb428422a32d56cd270c3b829720fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:48:41.215 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:41 smithi062 podman[38368]: 2024-04-17 07:48:41.143097508 +0000 UTC m=+0.331049144 container start 35a0481921de92e8361416940b0ae971cbb428422a32d56cd270c3b829720fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-17T07:48:41.215 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:41 smithi062 ceph-mgr[38387]: -- 172.21.15.62:0/3979554078 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641d82845a0 con 0x5641d8258c00 2024-04-17T07:48:41.215 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:41 smithi062 bash[38368]: 35a0481921de92e8361416940b0ae971cbb428422a32d56cd270c3b829720fba 2024-04-17T07:48:41.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:41 smithi053 ceph-mon[36437]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:41.553 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:41 smithi062 systemd[1]: Started Ceph mgr.z for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:48:42.053 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:41.704+0000 7fd921a52200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:48:42.053 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:41.800+0000 7fd921a52200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:48:42.553 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:42.473+0000 7fd921a52200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:48:42.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:42.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:48:42.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:42.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:42.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: Deploying daemon mgr.y on smithi053 2024-04-17T07:48:42.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-mon[36507]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:48:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:42.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:42.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: Deploying daemon mgr.y on smithi053 2024-04-17T07:48:42.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:42 smithi046 ceph-mon[32038]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:42.674 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.674 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.674 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:42.675 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:42.675 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:48:42.675 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:42.675 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:42.675 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: Deploying daemon mgr.y on smithi053 2024-04-17T07:48:42.675 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:42 smithi053 ceph-mon[36437]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:42.898 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:42.568+0000 7fd921a52200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:48:42.898 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:42.714+0000 7fd921a52200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:48:42.899 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:42.812+0000 7fd921a52200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:48:42.899 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:42.897+0000 7fd921a52200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:48:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T07:48:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T07:48:43.172 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-17T07:48:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-17T07:48:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0384306 s, 273 MB/s 2024-04-17T07:48:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-17T07:48:43.470 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:43.171+0000 7fd921a52200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:48:43.470 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:43.283+0000 7fd921a52200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:48:43.470 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:43.372+0000 7fd921a52200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:48:43.471 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:43.469+0000 7fd921a52200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:48:43.472 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 systemd[1]: Starting Ceph mgr.y for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:48:43.803 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:43.651+0000 7fd921a52200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:48:43.911 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 podman[37814]: 2024-04-17 07:48:43.618026842 +0000 UTC m=+0.094079102 container create 10ba0a837158b07ce5c56603d68373c20d62554a05dcc78505a3a3e9b22440f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-17T07:48:43.911 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 podman[37814]: 2024-04-17 07:48:43.542228911 +0000 UTC m=+0.018281151 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:48:43.911 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 podman[37814]: 2024-04-17 07:48:43.837041586 +0000 UTC m=+0.313093825 container init 10ba0a837158b07ce5c56603d68373c20d62554a05dcc78505a3a3e9b22440f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-17T07:48:43.911 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 podman[37814]: 2024-04-17 07:48:43.839619463 +0000 UTC m=+0.315671711 container start 10ba0a837158b07ce5c56603d68373c20d62554a05dcc78505a3a3e9b22440f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-17T07:48:43.911 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 ceph-mgr[37833]: -- 172.21.15.53:0/2227993662 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6c191c5a0 con 0x55c6c18f0800 2024-04-17T07:48:43.911 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 bash[37814]: 10ba0a837158b07ce5c56603d68373c20d62554a05dcc78505a3a3e9b22440f5 2024-04-17T07:48:43.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-04-17T07:48:44.221 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:43 smithi053 systemd[1]: Started Ceph mgr.y for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:48:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:48:44.721 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:44.407+0000 7f9897782200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:48:44.721 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:44.505+0000 7f9897782200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:48:44.982 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:44.685+0000 7fd921a52200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:48:44.982 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:44.835+0000 7fd921a52200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:48:45.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:44 smithi046 ceph-mon[32038]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:45.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:45.292 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:45.199+0000 7fd921a52200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:48:45.292 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:45.291+0000 7fd921a52200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:48:45.292 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-mon[36507]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:45.292 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.292 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.292 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.292 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.293 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:45.317 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:45.220+0000 7f9897782200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:48:45.317 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-mon[36437]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:45.318 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.318 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.318 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.318 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:45.318 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:45.553 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:45.509+0000 7fd921a52200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:48:45.567 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:45.317+0000 7f9897782200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:48:45.568 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:45.463+0000 7f9897782200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:48:45.568 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:45.567+0000 7f9897782200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:48:45.929 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:45.655+0000 7f9897782200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:48:46.003 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:45.705+0000 7fd921a52200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:48:46.003 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:46.002+0000 7fd921a52200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:48:46.221 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:45.929+0000 7f9897782200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:48:46.222 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:46.042+0000 7f9897782200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:48:46.222 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:46.131+0000 7f9897782200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:48:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.350+0000 7f5399600700 1 -- 172.21.15.46:0/172779063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394103cb0 msgr2=0x7f5394104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.350+0000 7f5399600700 1 --2- 172.21.15.46:0/172779063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394103cb0 0x7f5394104110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f537c0099c0 tx=0x7f537c009cd0 comp rx=0 tx=0).stop 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.350+0000 7f5399600700 1 -- 172.21.15.46:0/172779063 shutdown_connections 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.350+0000 7f5399600700 1 --2- 172.21.15.46:0/172779063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5394104650 0x7f539410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.350+0000 7f5399600700 1 --2- 172.21.15.46:0/172779063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5394102ab0 0x7f5394102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.350+0000 7f5399600700 1 --2- 172.21.15.46:0/172779063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394103cb0 0x7f5394104110 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.350+0000 7f5399600700 1 -- 172.21.15.46:0/172779063 >> 172.21.15.46:0/172779063 conn(0x7f53940fe050 msgr2=0x7f5394100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.351+0000 7f5399600700 1 -- 172.21.15.46:0/172779063 shutdown_connections 2024-04-17T07:48:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.351+0000 7f5399600700 1 -- 172.21.15.46:0/172779063 wait complete. 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 Processor -- start 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 -- start start 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394102ab0 0x7f539419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5394103cb0 0x7f539419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5394104650 0x7f53941a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f539410e6c0 con 0x7f5394104650 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f539410e540 con 0x7f5394102ab0 2024-04-17T07:48:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.352+0000 7f5399600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f539410e840 con 0x7f5394103cb0 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394102ab0 0x7f539419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f538a000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5394103cb0 0x7f539419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394102ab0 0x7f539419aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36218/0 (socket says 172.21.15.46:36218) 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f538a000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5394103cb0 0x7f539419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39722/0 (socket says 172.21.15.46:39722) 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 -- 172.21.15.46:0/2170285038 learned_addr learned my addr 172.21.15.46:0/2170285038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5393400700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5394104650 0x7f53941a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 -- 172.21.15.46:0/2170285038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5394103cb0 msgr2=0x7f539419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5394103cb0 0x7f539419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 -- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5394104650 msgr2=0x7f53941a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:48:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5394104650 0x7f53941a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 -- 172.21.15.46:0/2170285038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53941a2b50 con 0x7f5394102ab0 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5393400700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5394104650 0x7f53941a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.353+0000 7f5392a00700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394102ab0 0x7f539419aef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f538400d980 tx=0x7f538400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.354+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384016070 con 0x7f5394102ab0 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.354+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538400be30 con 0x7f5394102ab0 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.354+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53941a2e40 con 0x7f5394102ab0 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.354+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538400f3c0 con 0x7f5394102ab0 2024-04-17T07:48:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.354+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53941a32a0 con 0x7f5394102ab0 2024-04-17T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.355+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5394069770 con 0x7f5394102ab0 2024-04-17T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.356+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5384019070 con 0x7f5394102ab0 2024-04-17T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.356+0000 7f538be00700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5374039890 0x7f537403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.356+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f538404c6a0 con 0x7f5394102ab0 2024-04-17T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.356+0000 7f538a000700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5374039890 0x7f537403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:48:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.357+0000 7f538a000700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5374039890 0x7f537403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f537c000c00 tx=0x7f537c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:48:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.362+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5394069770 con 0x7f5394102ab0 2024-04-17T07:48:46.533 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:46.228+0000 7f9897782200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:48:46.533 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:46.410+0000 7f9897782200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:48:46.534 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:46.270+0000 7fd921a52200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:48:46.534 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:46.514+0000 7fd921a52200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:48:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:46.611+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f53940763a0 con 0x7f5374039890 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T07:48:46.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:46.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:46.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:46.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: Reconfiguring daemon mgr.x on smithi046 2024-04-17T07:48:46.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:46.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:46.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: Reconfiguring daemon mgr.x on smithi046 2024-04-17T07:48:46.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: Reconfiguring daemon mgr.x on smithi046 2024-04-17T07:48:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:47.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:47 smithi053 ceph-mon[36437]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:48:47.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:47 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:48:47.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:47 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:48:47.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:47 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:47.722 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:47.433+0000 7f9897782200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:48:47.722 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:47.569+0000 7f9897782200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:48:47.803 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:47.355+0000 7fd921a52200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:48:47.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:47 smithi062 ceph-mon[36507]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:48:47.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:47 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:48:47.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:47 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:48:47.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:47 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:47.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:47 smithi046 ceph-mon[32038]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:48:47.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:47 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:48:47.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:47 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:48:47.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:47 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:48.210 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:47.916+0000 7f9897782200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:48:48.210 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:48.008+0000 7f9897782200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:48:48.210 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:48.209+0000 7f9897782200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:48:48.290 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:47.966+0000 7fd921a52200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:48:48.291 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:48.067+0000 7fd921a52200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:48:48.291 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 07:48:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z[38383]: 2024-04-17T07:48:48.290+0000 7fd921a52200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:48:48.471 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:48.399+0000 7f9897782200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:48:48.959 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:48.692+0000 7f9897782200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:48:48.959 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:48.959+0000 7f9897782200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:48:49.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:49.043+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f538401b450 con 0x7f5394102ab0 2024-04-17T07:48:49.153 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:49.153 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.153 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.153 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:49.153 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:49.154 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:49.154 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.154 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: Standby manager daemon z started 2024-04-17T07:48:49.154 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T07:48:49.154 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:48:49.154 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T07:48:49.154 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:49 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:48:49.221 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:49.203+0000 7f9897782200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:48:49.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:49.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: Standby manager daemon z started 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T07:48:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:49 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:48:49.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:49.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:48:49.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:48:49.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:48:49.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:48:49.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: Standby manager daemon z started 2024-04-17T07:48:49.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T07:48:49.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:48:49.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T07:48:49.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:49 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.62:0/547406074' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:48:50.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:50 smithi046 ceph-mon[32038]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-17T07:48:50.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:50 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T07:48:50.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:50 smithi062 ceph-mon[36507]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-17T07:48:50.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:50 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T07:48:50.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:50 smithi053 ceph-mon[36437]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-17T07:48:50.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:50 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T07:48:50.472 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:50 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:50.037+0000 7f9897782200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:48:50.949 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:50 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:50.627+0000 7f9897782200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:48:50.949 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:50 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:50.727+0000 7f9897782200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:48:50.949 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 07:48:50 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y[37829]: 2024-04-17T07:48:50.949+0000 7f9897782200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:48:51.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:48:51.069+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mon.2 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5384042050 con 0x7f5394102ab0 2024-04-17T07:48:51.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:51 smithi053 ceph-mon[36437]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:51.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:51 smithi053 ceph-mon[36437]: Standby manager daemon y started 2024-04-17T07:48:51.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:51 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T07:48:51.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:51 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:48:51.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:51 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T07:48:51.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:51 smithi053 ceph-mon[36437]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:48:51.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:51 smithi062 ceph-mon[36507]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:51.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:51 smithi062 ceph-mon[36507]: Standby manager daemon y started 2024-04-17T07:48:51.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:51 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T07:48:51.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:51 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:48:51.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:51 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T07:48:51.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:51 smithi062 ceph-mon[36507]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:48:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:51 smithi046 ceph-mon[32038]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:51 smithi046 ceph-mon[32038]: Standby manager daemon y started 2024-04-17T07:48:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:51 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T07:48:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:51 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:48:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:51 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T07:48:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:51 smithi046 ceph-mon[32038]: from='mgr.? 172.21.15.53:0/1576933328' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:48:52.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-17T07:48:52.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T07:48:52.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1626261799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]: dispatch 2024-04-17T07:48:52.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]: dispatch 2024-04-17T07:48:52.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]': finished 2024-04-17T07:48:52.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T07:48:52.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:48:52.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:52 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2374613188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:48:52.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-17T07:48:52.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T07:48:52.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1626261799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]: dispatch 2024-04-17T07:48:52.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]: dispatch 2024-04-17T07:48:52.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]': finished 2024-04-17T07:48:52.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T07:48:52.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:48:52.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:52 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2374613188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1626261799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]: dispatch 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]: dispatch 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437464e9-4b20-4de4-9782-89a0daec7f70"}]': finished 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:48:52.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:52 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2374613188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:48:53.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:53 smithi046 ceph-mon[32038]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:53.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:53 smithi053 ceph-mon[36437]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:53.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:53 smithi062 ceph-mon[36507]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:55.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:55 smithi053 ceph-mon[36437]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:55.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:55 smithi062 ceph-mon[36507]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:55.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:55 smithi046 ceph-mon[32038]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:57.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:57 smithi046 ceph-mon[32038]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:57.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:57 smithi053 ceph-mon[36437]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:57 smithi062 ceph-mon[36507]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:59.401 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:48:59 smithi046 ceph-mon[32038]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:48:59 smithi053 ceph-mon[36437]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:48:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:48:59 smithi062 ceph-mon[36507]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:01.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:01 smithi046 ceph-mon[32038]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:01.452 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:01 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T07:49:01.452 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:01 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:01.452 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:01 smithi046 ceph-mon[32038]: Deploying daemon osd.0 on smithi046 2024-04-17T07:49:01.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:01 smithi053 ceph-mon[36437]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:01.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:01 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T07:49:01.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:01 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:01.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:01 smithi053 ceph-mon[36437]: Deploying daemon osd.0 on smithi046 2024-04-17T07:49:01.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:01 smithi062 ceph-mon[36507]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:01.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:01 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T07:49:01.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:01 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:01.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:01 smithi062 ceph-mon[36507]: Deploying daemon osd.0 on smithi046 2024-04-17T07:49:03.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:03 smithi046 ceph-mon[32038]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:03 smithi053 ceph-mon[36437]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:03 smithi062 ceph-mon[36507]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:05.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:05 smithi046 ceph-mon[32038]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:05.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:05 smithi053 ceph-mon[36437]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:05 smithi062 ceph-mon[36507]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:07.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:07 smithi046 ceph-mon[32038]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:07.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:07 smithi053 ceph-mon[36437]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:07.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:07 smithi062 ceph-mon[36507]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:09.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:09 smithi046 ceph-mon[32038]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:09.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:09.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:09.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:09.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:09 smithi053 ceph-mon[36437]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:09.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:09.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:09.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:09.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:09 smithi062 ceph-mon[36507]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:09.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:09.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:09.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:11.281 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:11 smithi046 ceph-mon[32038]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:11 smithi053 ceph-mon[36437]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:11.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:11 smithi062 ceph-mon[36507]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:12.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:12 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:12 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:12 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:12.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:12 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:12.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:12 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:12 smithi053 ceph-mon[36437]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:12 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:12 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:12 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:12 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:12 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:12 smithi062 ceph-mon[36507]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:12.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:12 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:12 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:12 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:12.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:12 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:12.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:12 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:12.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:12 smithi046 ceph-mon[32038]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.208+0000 7f538be00700 1 -- 172.21.15.46:0/2170285038 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53940763a0 con 0x7f5374039890 2024-04-17T07:49:13.208 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-04-17T07:49:13.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5374039890 msgr2=0x7f537403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:13.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5374039890 0x7f537403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f537c000c00 tx=0x7f537c005fb0 comp rx=0 tx=0).stop 2024-04-17T07:49:13.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394102ab0 msgr2=0x7f539419aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:13.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394102ab0 0x7f539419aef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f538400d980 tx=0x7f538400dc90 comp rx=0 tx=0).stop 2024-04-17T07:49:13.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 shutdown_connections 2024-04-17T07:49:13.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5374039890 0x7f537403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5394104650 0x7f53941a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5394102ab0 0x7f539419aef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 --2- 172.21.15.46:0/2170285038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5394103cb0 0x7f539419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 >> 172.21.15.46:0/2170285038 conn(0x7f53940fe050 msgr2=0x7f5394074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:49:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 shutdown_connections 2024-04-17T07:49:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:49:13.212+0000 7f5399600700 1 -- 172.21.15.46:0/2170285038 wait complete. 2024-04-17T07:49:13.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:13 smithi046 ceph-mon[32038]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T07:49:13.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:13.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:13.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:13 smithi062 ceph-mon[36507]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T07:49:13.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:13.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:13.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:13.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:13 smithi053 ceph-mon[36437]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T07:49:13.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:13.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:13.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:14.048 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.0.service 2024-04-17T07:49:14.053 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_4... 2024-04-17T07:49:14.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T07:49:14.314 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:49:14.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:14 smithi053 ceph-mon[36437]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T07:49:14.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:14 smithi053 ceph-mon[36437]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T07:49:14.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:14 smithi053 ceph-mon[36437]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:49:14.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:14 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:14.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:14 smithi053 ceph-mon[36437]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:14.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:14 smithi046 ceph-mon[32038]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T07:49:14.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:14 smithi046 ceph-mon[32038]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T07:49:14.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:14 smithi046 ceph-mon[32038]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:49:14.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:14 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:14.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:14 smithi046 ceph-mon[32038]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:14.530 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:49:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0[46987]: 2024-04-17T07:49:14.296+0000 7fb6c7600700 -1 osd.0 0 waiting for initial osdmap 2024-04-17T07:49:14.530 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:49:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0[46987]: 2024-04-17T07:49:14.304+0000 7fb6c2400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:49:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:14 smithi062 ceph-mon[36507]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T07:49:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:14 smithi062 ceph-mon[36507]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T07:49:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:14 smithi062 ceph-mon[36507]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:49:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:14 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:14 smithi062 ceph-mon[36507]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:15 smithi046 ceph-mon[32038]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:49:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:15 smithi046 ceph-mon[32038]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T07:49:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:15 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:15 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:15.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:15 smithi062 ceph-mon[36507]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:49:15.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:15 smithi062 ceph-mon[36507]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T07:49:15.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:15 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:15.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:15 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:15.621 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:15 smithi053 ceph-mon[36437]: from='osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:49:15.621 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:15 smithi053 ceph-mon[36437]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T07:49:15.621 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:15 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:15.621 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:15 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:16 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:49:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:16 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:49:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:16 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:16 smithi046 ceph-mon[32038]: osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] boot 2024-04-17T07:49:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:16 smithi046 ceph-mon[32038]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T07:49:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:16 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:16 smithi046 ceph-mon[32038]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:16.399 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T07:49:16.399 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T07:49:16.400 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-17T07:49:16.400 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-17T07:49:16.400 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00844063 s, 1.2 GB/s 2024-04-17T07:49:16.400 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-17T07:49:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:16 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:49:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:16 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:49:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:16 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:16 smithi062 ceph-mon[36507]: osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] boot 2024-04-17T07:49:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:16 smithi062 ceph-mon[36507]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T07:49:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:16 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:16.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:16 smithi062 ceph-mon[36507]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:16.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:16 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:49:16.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:16 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:49:16.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:16 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:16 smithi053 ceph-mon[36437]: osd.0 [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] boot 2024-04-17T07:49:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:16 smithi053 ceph-mon[36437]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T07:49:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:16 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:49:16.693 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:16 smithi053 ceph-mon[36437]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:49:16.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-04-17T07:49:17.221 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:49:17.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:17 smithi062 ceph-mon[36507]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T07:49:17.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:17 smithi053 ceph-mon[36437]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T07:49:17.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:17 smithi046 ceph-mon[32038]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T07:49:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:18 smithi046 ceph-mon[32038]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:18.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:18 smithi053 ceph-mon[36437]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:18.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 -- 172.21.15.53:0/1808047289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0103cb0 msgr2=0x7fd6f0104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:18.671 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/1808047289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0103cb0 0x7fd6f0104110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40099c0 tx=0x7fd6e4009cd0 comp rx=0 tx=0).stop 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 -- 172.21.15.53:0/1808047289 shutdown_connections 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/1808047289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f0104650 0x7fd6f010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/1808047289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f0102ab0 0x7fd6f0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/1808047289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0103cb0 0x7fd6f0104110 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 -- 172.21.15.53:0/1808047289 >> 172.21.15.53:0/1808047289 conn(0x7fd6f00fe050 msgr2=0x7fd6f0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 -- 172.21.15.53:0/1808047289 shutdown_connections 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.671+0000 7fd6f60c3700 1 -- 172.21.15.53:0/1808047289 wait complete. 2024-04-17T07:49:18.672 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 Processor -- start 2024-04-17T07:49:18.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 -- start start 2024-04-17T07:49:18.674 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f0102ab0 0x7fd6f00767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f0103cb0 0x7fd6f0074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0104650 0x7fd6f0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010e780 con 0x7fd6f0103cb0 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010e600 con 0x7fd6f0104650 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6f60c3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010e900 con 0x7fd6f0102ab0 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0104650 0x7fd6f0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6ef7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f0102ab0 0x7fd6f00767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0104650 0x7fd6f0075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50856/0 (socket says 172.21.15.53:50856) 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.672+0000 7fd6effff700 1 -- 172.21.15.53:0/3945981269 learned_addr learned my addr 172.21.15.53:0/3945981269 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ef7fe700 1 -- 172.21.15.53:0/3945981269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0104650 msgr2=0x7fd6f0075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ef7fe700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0104650 0x7fd6f0075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ef7fe700 1 -- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f0103cb0 msgr2=0x7fd6f0074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ef7fe700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f0103cb0 0x7fd6f0074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:18.675 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ef7fe700 1 -- 172.21.15.53:0/3945981269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f0075c30 con 0x7fd6f0102ab0 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6effff700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0104650 0x7fd6f0075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ef7fe700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f0102ab0 0x7fd6f00767f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0103b10 tx=0x7fd6e000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ecff9700 1 -- 172.21.15.53:0/3945981269 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0015020 con 0x7fd6f0102ab0 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ecff9700 1 -- 172.21.15.53:0/3945981269 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e0011d10 con 0x7fd6f0102ab0 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f01a2f20 con 0x7fd6f0102ab0 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6ecff9700 1 -- 172.21.15.53:0/3945981269 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0010d90 con 0x7fd6f0102ab0 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.673+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a3390 con 0x7fd6f0102ab0 2024-04-17T07:49:18.676 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.674+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f0069740 con 0x7fd6f0102ab0 2024-04-17T07:49:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.675+0000 7fd6ecff9700 1 -- 172.21.15.53:0/3945981269 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd6e0018020 con 0x7fd6f0102ab0 2024-04-17T07:49:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.676+0000 7fd6ecff9700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6d80a3c30 0x7fd6d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.676+0000 7fd6ecff9700 1 -- 172.21.15.53:0/3945981269 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fd6e00ce970 con 0x7fd6f0102ab0 2024-04-17T07:49:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.677+0000 7fd6eeffd700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6d80a3c30 0x7fd6d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:49:18.677 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.678+0000 7fd6eeffd700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6d80a3c30 0x7fd6d80a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4003790 tx=0x7fd6e4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:49:18.680 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.680+0000 7fd6ecff9700 1 -- 172.21.15.53:0/3945981269 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd6e0047070 con 0x7fd6f0102ab0 2024-04-17T07:49:18.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:18 smithi062 ceph-mon[36507]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:18.900 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:18.900+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd6f0062e90 con 0x7fd6d80a3c30 2024-04-17T07:49:19.831 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi046 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi046 to 17094M 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:49:19.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:20.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi046 2024-04-17T07:49:20.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi046 to 17094M 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:49:20.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:20.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi046 2024-04-17T07:49:20.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi046 to 17094M 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:49:20.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:20.871 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:20 smithi053 ceph-mon[36437]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:21.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:20 smithi046 ceph-mon[32038]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:21.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:20 smithi062 ceph-mon[36507]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:22.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:21 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/243040994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]: dispatch 2024-04-17T07:49:22.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:21 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]: dispatch 2024-04-17T07:49:22.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:21 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]': finished 2024-04-17T07:49:22.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:21 smithi046 ceph-mon[32038]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T07:49:22.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:21 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/243040994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]: dispatch 2024-04-17T07:49:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:21 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]: dispatch 2024-04-17T07:49:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:21 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]': finished 2024-04-17T07:49:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:21 smithi062 ceph-mon[36507]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T07:49:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:21 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/243040994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]: dispatch 2024-04-17T07:49:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:21 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]: dispatch 2024-04-17T07:49:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:21 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0511b8c-f523-4678-9683-ceaa6a1e65d1"}]': finished 2024-04-17T07:49:22.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:21 smithi053 ceph-mon[36437]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T07:49:22.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:23.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:22 smithi062 ceph-mon[36507]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:23.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:22 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/3010582621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:49:23.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:22 smithi046 ceph-mon[32038]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:23.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:22 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/3010582621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:49:23.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:22 smithi053 ceph-mon[36437]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:22 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/3010582621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:49:25.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:24 smithi062 ceph-mon[36507]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:25.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:24 smithi046 ceph-mon[32038]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:25.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:24 smithi053 ceph-mon[36437]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:27.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:26 smithi062 ceph-mon[36507]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:27.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:26 smithi046 ceph-mon[32038]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:27.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:26 smithi053 ceph-mon[36437]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:28.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:28 smithi053 ceph-mon[36437]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:28.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T07:49:28.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:28 smithi062 ceph-mon[36507]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T07:49:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:28 smithi046 ceph-mon[32038]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T07:49:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:29.947 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:29 smithi053 ceph-mon[36437]: Deploying daemon osd.1 on smithi053 2024-04-17T07:49:30.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:29 smithi062 ceph-mon[36507]: Deploying daemon osd.1 on smithi053 2024-04-17T07:49:30.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:29 smithi046 ceph-mon[32038]: Deploying daemon osd.1 on smithi053 2024-04-17T07:49:30.865 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:30 smithi053 ceph-mon[36437]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:31.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:30 smithi062 ceph-mon[36507]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:31.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:30 smithi046 ceph-mon[32038]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:33.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:32 smithi062 ceph-mon[36507]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:33.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:32 smithi046 ceph-mon[32038]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:33.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:32 smithi053 ceph-mon[36437]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:35 smithi053 ceph-mon[36437]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:35 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:35 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:35 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:35.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:35 smithi062 ceph-mon[36507]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:35.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:35 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:35.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:35 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:35.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:35 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:35.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:35 smithi046 ceph-mon[32038]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:35.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:35 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:35.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:35 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:35.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:35 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:37.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:37 smithi053 ceph-mon[36437]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:37.242 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-04-17T07:49:37.242 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.241+0000 7fd6ecff9700 1 -- 172.21.15.53:0/3945981269 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd6f0062e90 con 0x7fd6d80a3c30 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6d80a3c30 msgr2=0x7fd6d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6d80a3c30 0x7fd6d80a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4003790 tx=0x7fd6e4000bc0 comp rx=0 tx=0).stop 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f0102ab0 msgr2=0x7fd6f00767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f0102ab0 0x7fd6f00767f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0103b10 tx=0x7fd6e000b730 comp rx=0 tx=0).stop 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 shutdown_connections 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6d80a3c30 0x7fd6d80a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6f0104650 0x7fd6f0075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6f0102ab0 0x7fd6f00767f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:37.244 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 --2- 172.21.15.53:0/3945981269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f0103cb0 0x7fd6f0074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:37.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 >> 172.21.15.53:0/3945981269 conn(0x7fd6f00fe050 msgr2=0x7fd6f0106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:49:37.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 shutdown_connections 2024-04-17T07:49:37.245 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:49:37.244+0000 7fd6f60c3700 1 -- 172.21.15.53:0/3945981269 wait complete. 2024-04-17T07:49:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:37 smithi062 ceph-mon[36507]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:37.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:37 smithi046 ceph-mon[32038]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:37.890 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.1.service 2024-04-17T07:49:37.892 INFO:tasks.cephadm:Deploying osd.2 on smithi062 with /dev/vg_nvme/lv_4... 2024-04-17T07:49:37.892 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T07:49:38.137 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:49:38.181 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:49:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1[41028]: 2024-04-17T07:49:38.056+0000 7f0b1e6317c0 -1 osd.1 0 log_to_monitors true 2024-04-17T07:49:38.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:38 smithi062 ceph-mon[36507]: from='osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:49:38.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:38 smithi062 ceph-mon[36507]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:49:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:38 smithi053 ceph-mon[36437]: from='osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:49:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:38 smithi053 ceph-mon[36437]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:49:38.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:38.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:38 smithi046 ceph-mon[32038]: from='osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:49:38.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:38 smithi046 ceph-mon[32038]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:49:39.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:39 smithi062 ceph-mon[36507]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:39.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:39 smithi062 ceph-mon[36507]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T07:49:39.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:39 smithi062 ceph-mon[36507]: from='osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:49:39.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:39 smithi062 ceph-mon[36507]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T07:49:39.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:39.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:39 smithi062 ceph-mon[36507]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:49:39.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-mon[36437]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:39.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-mon[36437]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T07:49:39.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-mon[36437]: from='osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:49:39.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-mon[36437]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T07:49:39.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:39.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-mon[36437]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:49:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:39 smithi046 ceph-mon[32038]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:39 smithi046 ceph-mon[32038]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T07:49:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:39 smithi046 ceph-mon[32038]: from='osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:49:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:39 smithi046 ceph-mon[32038]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T07:49:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:39 smithi046 ceph-mon[32038]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:49:39.971 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1[41028]: 2024-04-17T07:49:39.538+0000 7f0b13c96700 -1 osd.1 0 waiting for initial osdmap 2024-04-17T07:49:39.971 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:49:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1[41028]: 2024-04-17T07:49:39.542+0000 7f0b0fa89700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:49:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T07:49:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T07:49:40.148 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-17T07:49:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-17T07:49:40.148 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111283 s, 942 MB/s 2024-04-17T07:49:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-17T07:49:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:40 smithi062 ceph-mon[36507]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:40 smithi062 ceph-mon[36507]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:49:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:40 smithi062 ceph-mon[36507]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T07:49:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:40.826 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:40 smithi053 ceph-mon[36437]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:40.826 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:40 smithi053 ceph-mon[36437]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:49:40.826 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:40 smithi053 ceph-mon[36437]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T07:49:40.826 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:40.826 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:40.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:40 smithi046 ceph-mon[32038]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:49:40.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:40 smithi046 ceph-mon[32038]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:49:40.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:40 smithi046 ceph-mon[32038]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T07:49:40.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:40.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:40.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-04-17T07:49:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:49:41.631 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] boot 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi053 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi053 to 17084M 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:41.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:41.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:41.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:41.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:41.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:49:41.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:49:41.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] boot 2024-04-17T07:49:41.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi053 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi053 to 17084M 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:41.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: osd.1 [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] boot 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi053 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi053 to 17084M 2024-04-17T07:49:41.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:41.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:41.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:41.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:49:41.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:41.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:49:41.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:49:42.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 -- 172.21.15.62:0/1346845865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 msgr2=0x7fbd9010af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:42.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 --2- 172.21.15.62:0/1346845865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd9010af20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b0a0 tx=0x7fbd8800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:49:42.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 -- 172.21.15.62:0/1346845865 shutdown_connections 2024-04-17T07:49:42.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 --2- 172.21.15.62:0/1346845865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd9010af20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:42.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 --2- 172.21.15.62:0/1346845865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd90102ad0 0x7fbd90102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:42.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 --2- 172.21.15.62:0/1346845865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd90103cd0 0x7fbd90104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 -- 172.21.15.62:0/1346845865 >> 172.21.15.62:0/1346845865 conn(0x7fbd900fe070 msgr2=0x7fbd901004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 -- 172.21.15.62:0/1346845865 shutdown_connections 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 -- 172.21.15.62:0/1346845865 wait complete. 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.761+0000 7fbd961da700 1 Processor -- start 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd961da700 1 -- start start 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd961da700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd90102ad0 0x7fbd90198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd961da700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd90103cd0 0x7fbd90199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd961da700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd901a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd961da700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9010e750 con 0x7fbd90102ad0 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd961da700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9010e5d0 con 0x7fbd90103cd0 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd961da700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9010e8d0 con 0x7fbd90104670 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd901a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd901a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49420/0 (socket says 172.21.15.62:49420) 2024-04-17T07:49:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd90103cd0 0x7fbd90199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 -- 172.21.15.62:0/460248713 learned_addr learned my addr 172.21.15.62:0/460248713 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 -- 172.21.15.62:0/460248713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd90103cd0 msgr2=0x7fbd90199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd90103cd0 0x7fbd90199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 -- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd90102ad0 msgr2=0x7fbd90198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd90102ad0 0x7fbd90198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 -- 172.21.15.62:0/460248713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd901a0950 con 0x7fbd90104670 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8ffff700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd901a0280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b070 tx=0x7fbd88009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.762+0000 7fbd8cff9700 1 -- 172.21.15.62:0/460248713 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88021070 con 0x7fbd90104670 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.763+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd901a0be0 con 0x7fbd90104670 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.763+0000 7fbd8cff9700 1 -- 172.21.15.62:0/460248713 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd88007d40 con 0x7fbd90104670 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.763+0000 7fbd8cff9700 1 -- 172.21.15.62:0/460248713 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8801b850 con 0x7fbd90104670 2024-04-17T07:49:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.763+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd901a1090 con 0x7fbd90104670 2024-04-17T07:49:42.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.764+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd90069740 con 0x7fbd90104670 2024-04-17T07:49:42.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.765+0000 7fbd8cff9700 1 -- 172.21.15.62:0/460248713 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fbd88024070 con 0x7fbd90104670 2024-04-17T07:49:42.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.766+0000 7fbd8cff9700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbd780a3bb0 0x7fbd780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:49:42.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.766+0000 7fbd8cff9700 1 -- 172.21.15.62:0/460248713 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fbd8800f020 con 0x7fbd90104670 2024-04-17T07:49:42.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.767+0000 7fbd8f7fe700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbd780a3bb0 0x7fbd780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:49:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.768+0000 7fbd8f7fe700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbd780a3bb0 0x7fbd780a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd9019a1e0 tx=0x7fbd84006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:49:42.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.770+0000 7fbd8cff9700 1 -- 172.21.15.62:0/460248713 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbd8802b020 con 0x7fbd90104670 2024-04-17T07:49:42.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:49:42.989+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbd90062e90 con 0x7fbd780a3bb0 2024-04-17T07:49:43.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:42 smithi053 ceph-mon[36437]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:43.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:42 smithi053 ceph-mon[36437]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T07:49:43.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:42 smithi062 ceph-mon[36507]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:43.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:42 smithi062 ceph-mon[36507]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T07:49:43.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:42 smithi046 ceph-mon[32038]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:43.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:42 smithi046 ceph-mon[32038]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T07:49:44.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:43 smithi053 ceph-mon[36437]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:49:44.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:49:44.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:49:44.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:44.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:43 smithi062 ceph-mon[36507]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:49:44.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:49:44.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:49:44.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:43 smithi046 ceph-mon[32038]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:49:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:49:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:49:44.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:45.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:44 smithi062 ceph-mon[36507]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:45.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:44 smithi053 ceph-mon[36437]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:45.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:44 smithi046 ceph-mon[32038]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:46.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:45 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/63654549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]: dispatch 2024-04-17T07:49:46.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:45 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]: dispatch 2024-04-17T07:49:46.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:45 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]': finished 2024-04-17T07:49:46.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:45 smithi062 ceph-mon[36507]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T07:49:46.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:45 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:49:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:45 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/63654549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]: dispatch 2024-04-17T07:49:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:45 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]: dispatch 2024-04-17T07:49:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:45 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]': finished 2024-04-17T07:49:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:45 smithi046 ceph-mon[32038]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T07:49:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:45 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:49:46.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:45 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/63654549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]: dispatch 2024-04-17T07:49:46.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:45 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]: dispatch 2024-04-17T07:49:46.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:45 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246"}]': finished 2024-04-17T07:49:46.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:45 smithi053 ceph-mon[36437]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T07:49:46.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:45 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:49:47.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:46 smithi062 ceph-mon[36507]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:47.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:46 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/3237862877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:49:47.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:46 smithi046 ceph-mon[32038]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:47.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:46 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/3237862877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:49:47.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:46 smithi053 ceph-mon[36437]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:47.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:46 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/3237862877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:49:49.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:48 smithi062 ceph-mon[36507]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:49.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:48 smithi046 ceph-mon[32038]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:49.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:48 smithi053 ceph-mon[36437]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:51.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:50 smithi062 ceph-mon[36507]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:50 smithi046 ceph-mon[32038]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:51.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:50 smithi053 ceph-mon[36437]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:53.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:52 smithi062 ceph-mon[36507]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:53.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:52 smithi046 ceph-mon[32038]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:53.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:52 smithi053 ceph-mon[36437]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T07:49:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:54.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T07:49:54.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:54.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T07:49:54.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:49:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:55 smithi062 ceph-mon[36507]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:55 smithi062 ceph-mon[36507]: Deploying daemon osd.2 on smithi062 2024-04-17T07:49:55.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:55 smithi046 ceph-mon[32038]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:55.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:55 smithi046 ceph-mon[32038]: Deploying daemon osd.2 on smithi062 2024-04-17T07:49:55.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:55 smithi053 ceph-mon[36437]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:55.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:55 smithi053 ceph-mon[36437]: Deploying daemon osd.2 on smithi062 2024-04-17T07:49:57.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:57 smithi062 ceph-mon[36507]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:57.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:57 smithi046 ceph-mon[32038]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:57.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:57 smithi053 ceph-mon[36437]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:59.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:49:59 smithi062 ceph-mon[36507]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:59.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:49:59 smithi046 ceph-mon[32038]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:49:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:49:59 smithi053 ceph-mon[36437]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:00.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:00.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:00.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:00.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:00 smithi062 ceph-mon[36507]: overall HEALTH_OK 2024-04-17T07:50:00.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:00.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:00.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:00.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:00 smithi046 ceph-mon[32038]: overall HEALTH_OK 2024-04-17T07:50:00.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:00.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:00.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:00.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:00 smithi053 ceph-mon[36437]: overall HEALTH_OK 2024-04-17T07:50:01.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:01 smithi046 ceph-mon[32038]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:01.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:01 smithi062 ceph-mon[36507]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:01.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:01 smithi053 ceph-mon[36437]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:02.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:02 smithi062 ceph-mon[36507]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:02.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:02.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:02.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:02.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:02.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:02.928 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 2 on host 'smithi062' 2024-04-17T07:50:02.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.928+0000 7fbd8cff9700 1 -- 172.21.15.62:0/460248713 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd90062e90 con 0x7fbd780a3bb0 2024-04-17T07:50:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbd780a3bb0 msgr2=0x7fbd780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbd780a3bb0 0x7fbd780a6070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd9019a1e0 tx=0x7fbd84006d20 comp rx=0 tx=0).stop 2024-04-17T07:50:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 msgr2=0x7fbd901a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd901a0280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b070 tx=0x7fbd88009800 comp rx=0 tx=0).stop 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 shutdown_connections 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbd780a3bb0 0x7fbd780a6070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd90104670 0x7fbd901a0280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd90102ad0 0x7fbd90198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 --2- 172.21.15.62:0/460248713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd90103cd0 0x7fbd90199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 >> 172.21.15.62:0/460248713 conn(0x7fbd900fe070 msgr2=0x7fbd90106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 shutdown_connections 2024-04-17T07:50:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:50:02.932+0000 7fbd961da700 1 -- 172.21.15.62:0/460248713 wait complete. 2024-04-17T07:50:03.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:02 smithi046 ceph-mon[32038]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:03.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:03.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:03.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:02 smithi053 ceph-mon[36437]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.567 DEBUG:teuthology.orchestra.run.smithi062:osd.2> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.2.service 2024-04-17T07:50:03.569 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_3... 2024-04-17T07:50:03.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T07:50:03.931 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.931 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:03.932 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:03.932 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:03 smithi046 ceph-mon[32038]: from='osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:50:03.932 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:03 smithi046 ceph-mon[32038]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:50:04.044 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:50:04.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:03 smithi053 ceph-mon[36437]: from='osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:50:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:03 smithi053 ceph-mon[36437]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:50:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:03 smithi062 ceph-mon[36507]: from='osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:50:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:03 smithi062 ceph-mon[36507]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:50:05.050 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:04 smithi062 ceph-mon[36507]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:05.050 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:04 smithi062 ceph-mon[36507]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T07:50:05.050 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:04 smithi062 ceph-mon[36507]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T07:50:05.050 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:05.051 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:04 smithi062 ceph-mon[36507]: from='osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:50:05.051 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:04 smithi062 ceph-mon[36507]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:50:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:04 smithi053 ceph-mon[36437]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:04 smithi053 ceph-mon[36437]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T07:50:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:04 smithi053 ceph-mon[36437]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T07:50:05.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:05.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:04 smithi053 ceph-mon[36437]: from='osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:50:05.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:04 smithi053 ceph-mon[36437]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:50:05.268 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:04 smithi046 ceph-mon[32038]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:05.268 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:04 smithi046 ceph-mon[32038]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T07:50:05.268 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:04 smithi046 ceph-mon[32038]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T07:50:05.268 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:05.268 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:04 smithi046 ceph-mon[32038]: from='osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:50:05.268 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:04 smithi046 ceph-mon[32038]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:50:05.924 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2[41678]: 2024-04-17T07:50:05.670+0000 7f4532af3700 -1 osd.2 0 waiting for initial osdmap 2024-04-17T07:50:05.924 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2[41678]: 2024-04-17T07:50:05.676+0000 7f452a0dd700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:50:06.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:05 smithi053 ceph-mon[36437]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:50:06.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:05 smithi053 ceph-mon[36437]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T07:50:06.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:06.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:06.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:05 smithi053 ceph-mon[36437]: from='osd.2 ' entity='osd.2' 2024-04-17T07:50:06.285 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:05 smithi062 ceph-mon[36507]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:50:06.285 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:05 smithi062 ceph-mon[36507]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T07:50:06.285 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:06.285 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:06.285 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:05 smithi062 ceph-mon[36507]: from='osd.2 ' entity='osd.2' 2024-04-17T07:50:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:05 smithi046 ceph-mon[32038]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:50:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:05 smithi046 ceph-mon[32038]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T07:50:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:05 smithi046 ceph-mon[32038]: from='osd.2 ' entity='osd.2' 2024-04-17T07:50:06.829 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T07:50:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T07:50:06.830 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-17T07:50:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-17T07:50:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0628658 s, 167 MB/s 2024-04-17T07:50:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-17T07:50:07.108 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:50:07.108 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:50:07.108 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] boot 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:07.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] boot 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:50:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:50:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:50:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: osd.2 [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] boot 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:07.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:07.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-04-17T07:50:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:07 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi062 2024-04-17T07:50:08.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:07 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi062 to 17080M 2024-04-17T07:50:08.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:50:08.156 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:50:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:07 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi062 2024-04-17T07:50:08.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:07 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi062 to 17080M 2024-04-17T07:50:08.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:07 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:50:08.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:07 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi062 2024-04-17T07:50:08.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:07 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi062 to 17080M 2024-04-17T07:50:08.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:07 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:50:09.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:08 smithi053 ceph-mon[36437]: pgmap v91: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:09.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:08 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T07:50:09.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:08 smithi053 ceph-mon[36437]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T07:50:09.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:08 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:50:09.226 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51205]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:50:09.226 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.226 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51205]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.226 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51205]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:08 smithi046 ceph-mon[32038]: pgmap v91: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:09.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T07:50:09.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:08 smithi046 ceph-mon[32038]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T07:50:09.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:50:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:08 smithi062 ceph-mon[36507]: pgmap v91: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:08 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T07:50:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:08 smithi062 ceph-mon[36507]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T07:50:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:08 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:50:09.483 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51208]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:50:09.483 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.484 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51208]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.484 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51208]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44741]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44741]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44741]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44741]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44744]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44744]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44744]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.613 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44744]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51211]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51211]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51211]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51216]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 sudo[51216]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45471]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45471]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45471]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45471]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45474]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45474]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45474]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.803 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45474]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45477]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45477]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45477]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45477]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45480]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45480]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45480]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 sudo[45480]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44747]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:50:09.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44747]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44747]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:09.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44750]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:50:09.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:50:09.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44750]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:50:09.956 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 sudo[44750]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:50:10.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T07:50:10.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:09 smithi046 ceph-mon[32038]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T07:50:10.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:09 smithi053 ceph-mon[36437]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.223+0000 7f085a600700 1 -- 172.21.15.46:0/4044108828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410ac90 msgr2=0x7f0854111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.223+0000 7f085a600700 1 --2- 172.21.15.46:0/4044108828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410ac90 0x7f0854111540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b0d0 tx=0x7f084c00b3e0 comp rx=0 tx=0).stop 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.224+0000 7f085a600700 1 -- 172.21.15.46:0/4044108828 shutdown_connections 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.224+0000 7f085a600700 1 --2- 172.21.15.46:0/4044108828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410ac90 0x7f0854111540 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.224+0000 7f085a600700 1 --2- 172.21.15.46:0/4044108828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 0x7f0854109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.224+0000 7f085a600700 1 --2- 172.21.15.46:0/4044108828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085410a2f0 0x7f085410a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.224+0000 7f085a600700 1 -- 172.21.15.46:0/4044108828 >> 172.21.15.46:0/4044108828 conn(0x7f08540780c0 msgr2=0x7f085407a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.224+0000 7f085a600700 1 -- 172.21.15.46:0/4044108828 shutdown_connections 2024-04-17T07:50:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.224+0000 7f085a600700 1 -- 172.21.15.46:0/4044108828 wait complete. 2024-04-17T07:50:10.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 Processor -- start 2024-04-17T07:50:10.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 -- start start 2024-04-17T07:50:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 0x7f085419f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410a2f0 0x7f085419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085410ac90 0x7f08541a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0854114ce0 con 0x7f085410a2f0 2024-04-17T07:50:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0854114b60 con 0x7f085410ac90 2024-04-17T07:50:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.225+0000 7f085a600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0854114e60 con 0x7f08541090f0 2024-04-17T07:50:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 0x7f085419f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 0x7f085419f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:48054/0 (socket says 172.21.15.46:48054) 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 -- 172.21.15.46:0/2227383020 learned_addr learned my addr 172.21.15.46:0/2227383020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0852a00700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410a2f0 0x7f085419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853e00700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085410ac90 0x7f08541a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 -- 172.21.15.46:0/2227383020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085410ac90 msgr2=0x7f08541a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085410ac90 0x7f08541a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 -- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410a2f0 msgr2=0x7f085419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410a2f0 0x7f085419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.226+0000 7f0853400700 1 -- 172.21.15.46:0/2227383020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08541a6e70 con 0x7f08541090f0 2024-04-17T07:50:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.227+0000 7f0853400700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 0x7f085419f210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f084400e980 tx=0x7f084400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:50:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.227+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844004230 con 0x7f08541090f0 2024-04-17T07:50:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.227+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08440043d0 con 0x7f08541090f0 2024-04-17T07:50:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.227+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844003dc0 con 0x7f08541090f0 2024-04-17T07:50:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.227+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08541a7160 con 0x7f08541090f0 2024-04-17T07:50:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.227+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08541a76a0 con 0x7f08541090f0 2024-04-17T07:50:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.228+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0854069740 con 0x7f08541090f0 2024-04-17T07:50:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.230+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f084400c810 con 0x7f08541090f0 2024-04-17T07:50:10.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.232+0000 7f084be00700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f08380a3cb0 0x7f08380a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.232+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f08440cf330 con 0x7f08541090f0 2024-04-17T07:50:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.232+0000 7f0852a00700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f08380a3cb0 0x7f08380a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:10.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.233+0000 7f0852a00700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f08380a3cb0 0x7f08380a6170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0840009c60 tx=0x7f0840009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:50:10.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.234+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0844044e20 con 0x7f08541090f0 2024-04-17T07:50:10.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T07:50:10.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T07:50:10.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:50:10.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:50:10.308 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:50:10.308 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:50:10.308 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:09 smithi062 ceph-mon[36507]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:50:10.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.467+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0844099090 con 0x7f08541090f0 2024-04-17T07:50:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:10.487+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0854062e90 con 0x7f08380a3cb0 2024-04-17T07:50:11.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:10 smithi053 ceph-mon[36437]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:11.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:10 smithi053 ceph-mon[36437]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T07:50:11.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:10 smithi053 ceph-mon[36437]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-17T07:50:11.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:50:11.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:50:11.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:10 smithi062 ceph-mon[36507]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:10 smithi062 ceph-mon[36507]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T07:50:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:10 smithi062 ceph-mon[36507]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-17T07:50:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:50:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:50:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:11.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:10 smithi046 ceph-mon[32038]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:11.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:10 smithi046 ceph-mon[32038]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T07:50:11.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:10 smithi046 ceph-mon[32038]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-17T07:50:11.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:50:11.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:50:11.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:12.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:11 smithi046 ceph-mon[32038]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:50:12.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:11 smithi053 ceph-mon[36437]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:50:12.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:11 smithi062 ceph-mon[36507]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:50:13.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:12 smithi046 ceph-mon[32038]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:12 smithi053 ceph-mon[36437]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:13.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:12 smithi062 ceph-mon[36507]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:15.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:14 smithi062 ceph-mon[36507]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:15.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:14 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3393662267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "398d84f2-695b-4446-872d-55ba26ab8c94"}]: dispatch 2024-04-17T07:50:15.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:14 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3393662267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "398d84f2-695b-4446-872d-55ba26ab8c94"}]': finished 2024-04-17T07:50:15.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:14 smithi062 ceph-mon[36507]: osdmap e22: 4 total, 3 up, 4 in 2024-04-17T07:50:15.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:14 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:14 smithi046 ceph-mon[32038]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:14 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3393662267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "398d84f2-695b-4446-872d-55ba26ab8c94"}]: dispatch 2024-04-17T07:50:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:14 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3393662267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "398d84f2-695b-4446-872d-55ba26ab8c94"}]': finished 2024-04-17T07:50:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:14 smithi046 ceph-mon[32038]: osdmap e22: 4 total, 3 up, 4 in 2024-04-17T07:50:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:14 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:15.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:14 smithi053 ceph-mon[36437]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:15.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:14 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3393662267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "398d84f2-695b-4446-872d-55ba26ab8c94"}]: dispatch 2024-04-17T07:50:15.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:14 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3393662267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "398d84f2-695b-4446-872d-55ba26ab8c94"}]': finished 2024-04-17T07:50:15.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:14 smithi053 ceph-mon[36437]: osdmap e22: 4 total, 3 up, 4 in 2024-04-17T07:50:15.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:14 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:16.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:15 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/942142827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:50:16.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:15 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/942142827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:50:16.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:15 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/942142827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:50:17.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:16 smithi062 ceph-mon[36507]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:17.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:16 smithi046 ceph-mon[32038]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:17.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:16 smithi053 ceph-mon[36437]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:19.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:18 smithi046 ceph-mon[32038]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:19.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:18 smithi062 ceph-mon[36507]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:19.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:18 smithi053 ceph-mon[36437]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:20 smithi062 ceph-mon[36507]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:21.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:20 smithi046 ceph-mon[32038]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:21.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:20 smithi053 ceph-mon[36437]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:23.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:22 smithi046 ceph-mon[32038]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:22 smithi062 ceph-mon[36507]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:22 smithi053 ceph-mon[36437]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T07:50:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T07:50:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:24.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T07:50:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:25 smithi062 ceph-mon[36507]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:25 smithi062 ceph-mon[36507]: Deploying daemon osd.3 on smithi046 2024-04-17T07:50:25.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:25 smithi046 ceph-mon[32038]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:25.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:25 smithi046 ceph-mon[32038]: Deploying daemon osd.3 on smithi046 2024-04-17T07:50:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:25 smithi053 ceph-mon[36437]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:25 smithi053 ceph-mon[36437]: Deploying daemon osd.3 on smithi046 2024-04-17T07:50:27.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:27 smithi046 ceph-mon[32038]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:27.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:27 smithi062 ceph-mon[36507]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:27 smithi053 ceph-mon[36437]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:29 smithi062 ceph-mon[36507]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:29.339 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:29 smithi046 ceph-mon[32038]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:29 smithi053 ceph-mon[36437]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:31.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:31 smithi062 ceph-mon[36507]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:31.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:31 smithi046 ceph-mon[32038]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:31 smithi053 ceph-mon[36437]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:32.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:32.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:32.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:32.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:32.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:32.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:32.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:32.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:32.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:33.252 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:33 smithi046 ceph-mon[32038]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:33.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:33 smithi062 ceph-mon[36507]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:33.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:33 smithi053 ceph-mon[36437]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:35.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:35 smithi062 ceph-mon[36507]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:35.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:35 smithi046 ceph-mon[32038]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:35.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:35 smithi053 ceph-mon[36437]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:36.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:36.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:36.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:36.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: from='osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:50:36.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:36 smithi046 ceph-mon[32038]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:50:36.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: from='osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:50:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:36 smithi062 ceph-mon[36507]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:50:36.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:36.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:36.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:36.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:36.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: from='osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:50:36.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:36 smithi053 ceph-mon[36437]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:50:36.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.937+0000 7f084be00700 1 -- 172.21.15.46:0/2227383020 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0854062e90 con 0x7f08380a3cb0 2024-04-17T07:50:36.939 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-04-17T07:50:36.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f08380a3cb0 msgr2=0x7f08380a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:36.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f08380a3cb0 0x7f08380a6170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0840009c60 tx=0x7f0840009260 comp rx=0 tx=0).stop 2024-04-17T07:50:36.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 msgr2=0x7f085419f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:36.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 0x7f085419f210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f084400e980 tx=0x7f084400ec90 comp rx=0 tx=0).stop 2024-04-17T07:50:36.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 shutdown_connections 2024-04-17T07:50:36.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f08380a3cb0 0x7f08380a6170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f085410ac90 0x7f08541a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08541090f0 0x7f085419f210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 --2- 172.21.15.46:0/2227383020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085410a2f0 0x7f085419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.941+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 >> 172.21.15.46:0/2227383020 conn(0x7f08540780c0 msgr2=0x7f085410d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:50:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.942+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 shutdown_connections 2024-04-17T07:50:36.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:50:36.942+0000 7f085a600700 1 -- 172.21.15.46:0/2227383020 wait complete. 2024-04-17T07:50:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T07:50:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: from='osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:50:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T07:50:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:50:37.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:37.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:37.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:37 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:37.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T07:50:37.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: from='osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:50:37.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T07:50:37.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:37.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:50:37.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:37.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:37.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:37.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T07:50:37.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: from='osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:50:37.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T07:50:37.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:37.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:50:37.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:37.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:37.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:37 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:37.756 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.3.service 2024-04-17T07:50:37.761 INFO:tasks.cephadm:Deploying osd.4 on smithi053 with /dev/vg_nvme/lv_3... 2024-04-17T07:50:37.761 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T07:50:38.019 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:50:38.228 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T07:50:37.975+0000 7fac82800700 -1 osd.3 0 waiting for initial osdmap 2024-04-17T07:50:38.229 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 07:50:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T07:50:37.991+0000 7fac78600700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: from='osd.3 ' entity='osd.3' 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] boot 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: osdmap e25: 4 total, 4 up, 4 in 2024-04-17T07:50:38.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:50:38.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T07:50:38.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:38.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: from='osd.3 ' entity='osd.3' 2024-04-17T07:50:38.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] boot 2024-04-17T07:50:38.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: osdmap e25: 4 total, 4 up, 4 in 2024-04-17T07:50:38.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:50:38.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T07:50:38.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:38.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:50:38.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: from='osd.3 ' entity='osd.3' 2024-04-17T07:50:38.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: osd.3 [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] boot 2024-04-17T07:50:38.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: osdmap e25: 4 total, 4 up, 4 in 2024-04-17T07:50:38.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:50:39.390 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:39 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:50:39.390 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:39 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:50:39.390 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:39 smithi053 ceph-mon[36437]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T07:50:39.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:39 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:50:39.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:39 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:50:39.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:39 smithi046 ceph-mon[32038]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T07:50:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:39 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:50:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:39 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:50:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:39 smithi062 ceph-mon[36507]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T07:50:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T07:50:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T07:50:40.252 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-17T07:50:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-17T07:50:40.252 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00835079 s, 1.3 GB/s 2024-04-17T07:50:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-17T07:50:40.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:40 smithi062 ceph-mon[36507]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:50:40.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:40 smithi053 ceph-mon[36437]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:50:40.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:40 smithi046 ceph-mon[32038]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:50:40.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-04-17T07:50:41.201 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:50:41.524 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:41 smithi053 ceph-mon[36437]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T07:50:41.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:41 smithi062 ceph-mon[36507]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T07:50:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:41 smithi046 ceph-mon[32038]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T07:50:42.498 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:42 smithi053 ceph-mon[36437]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T07:50:42.498 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:42 smithi053 ceph-mon[36437]: pgmap v118: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:42.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:42 smithi046 ceph-mon[32038]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T07:50:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:42 smithi046 ceph-mon[32038]: pgmap v118: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:42.739 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 -- 172.21.15.53:0/123410355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400102bb0 msgr2=0x7fb400103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 --2- 172.21.15.53:0/123410355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400102bb0 0x7fb400103010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0099c0 tx=0x7fb3ec009cd0 comp rx=0 tx=0).stop 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 -- 172.21.15.53:0/123410355 shutdown_connections 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 --2- 172.21.15.53:0/123410355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb400103550 0x7fb400109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 --2- 172.21.15.53:0/123410355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4001019b0 0x7fb400101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 --2- 172.21.15.53:0/123410355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400102bb0 0x7fb400103010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 -- 172.21.15.53:0/123410355 >> 172.21.15.53:0/123410355 conn(0x7fb4000fcf50 msgr2=0x7fb4000ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 -- 172.21.15.53:0/123410355 shutdown_connections 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.731+0000 7fb406b31700 1 -- 172.21.15.53:0/123410355 wait complete. 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 Processor -- start 2024-04-17T07:50:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 -- start start 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4001019b0 0x7fb400105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb400102bb0 0x7fb400103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400103550 0x7fb4001044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb40010d570 con 0x7fb400102bb0 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb40010d3f0 con 0x7fb400103550 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb406b31700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb40010d6f0 con 0x7fb4001019b0 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400103550 0x7fb4001044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400103550 0x7fb4001044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45866/0 (socket says 172.21.15.53:45866) 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 -- 172.21.15.53:0/1064991285 learned_addr learned my addr 172.21.15.53:0/1064991285 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4048cd700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4001019b0 0x7fb400105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 -- 172.21.15.53:0/1064991285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4001019b0 msgr2=0x7fb400105940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb3fffff700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb400102bb0 0x7fb400103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4001019b0 0x7fb400105940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 -- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb400102bb0 msgr2=0x7fb400103f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb400102bb0 0x7fb400103f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb4050ce700 1 -- 172.21.15.53:0/1064991285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb400104d80 con 0x7fb400103550 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.732+0000 7fb3fffff700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb400102bb0 0x7fb400103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.733+0000 7fb4050ce700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400103550 0x7fb4001044d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800ada0 tx=0x7fb3f800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.733+0000 7fb3fdffb700 1 -- 172.21.15.53:0/1064991285 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8017020 con 0x7fb400103550 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.733+0000 7fb3fdffb700 1 -- 172.21.15.53:0/1064991285 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f800a0d0 con 0x7fb400103550 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.733+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4001a2e60 con 0x7fb400103550 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.733+0000 7fb3fdffb700 1 -- 172.21.15.53:0/1064991285 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8005480 con 0x7fb400103550 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.733+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4001a32c0 con 0x7fb400103550 2024-04-17T07:50:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.734+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb400069740 con 0x7fb400103550 2024-04-17T07:50:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.735+0000 7fb3fdffb700 1 -- 172.21.15.53:0/1064991285 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3f8005620 con 0x7fb400103550 2024-04-17T07:50:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.735+0000 7fb3fdffb700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb3f00a3c40 0x7fb3f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:50:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.736+0000 7fb3fdffb700 1 -- 172.21.15.53:0/1064991285 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fb3f80ce9a0 con 0x7fb400103550 2024-04-17T07:50:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.736+0000 7fb4048cd700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb3f00a3c40 0x7fb3f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:50:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.736+0000 7fb4048cd700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb3f00a3c40 0x7fb3f00a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb400102a10 tx=0x7fb3f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:50:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.740+0000 7fb3fdffb700 1 -- 172.21.15.53:0/1064991285 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb3f8021020 con 0x7fb400103550 2024-04-17T07:50:42.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:42 smithi062 ceph-mon[36507]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T07:50:42.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:42 smithi062 ceph-mon[36507]: pgmap v118: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:42.955 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:50:42.955+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb4000ff190 con 0x7fb3f00a3c40 2024-04-17T07:50:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:43 smithi046 ceph-mon[32038]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-17T07:50:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:43 smithi046 ceph-mon[32038]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T07:50:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:43 smithi046 ceph-mon[32038]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:50:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:50:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:50:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:43.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:43 smithi053 ceph-mon[36437]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-17T07:50:43.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:43 smithi053 ceph-mon[36437]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T07:50:43.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:43 smithi053 ceph-mon[36437]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:50:43.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:50:43.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:50:43.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:43.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:43 smithi062 ceph-mon[36507]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-17T07:50:43.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:43 smithi062 ceph-mon[36507]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T07:50:43.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:43 smithi062 ceph-mon[36507]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:50:43.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:50:43.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:50:43.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:44.850 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: pgmap v120: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:44.850 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi046 2024-04-17T07:50:44.850 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:44.850 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:44.850 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:44.850 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:44.850 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi046 to 8547M 2024-04-17T07:50:44.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:44.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:44.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:44.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:44 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: pgmap v120: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi046 2024-04-17T07:50:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:45.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:45.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi046 to 8547M 2024-04-17T07:50:45.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:45.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:45.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:44 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: pgmap v120: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:45.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi046 2024-04-17T07:50:45.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:45.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:50:45.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi046 to 8547M 2024-04-17T07:50:45.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:45.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:45.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:50:45.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:44 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:46 smithi062 ceph-mon[36507]: pgmap v121: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:46 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/244337252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]: dispatch 2024-04-17T07:50:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:46 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]: dispatch 2024-04-17T07:50:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:46 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]': finished 2024-04-17T07:50:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:46 smithi062 ceph-mon[36507]: osdmap e30: 5 total, 4 up, 5 in 2024-04-17T07:50:46.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:50:46.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:46 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/1207503685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:50:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:46 smithi053 ceph-mon[36437]: pgmap v121: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:46 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/244337252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]: dispatch 2024-04-17T07:50:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:46 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]: dispatch 2024-04-17T07:50:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:46 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]': finished 2024-04-17T07:50:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:46 smithi053 ceph-mon[36437]: osdmap e30: 5 total, 4 up, 5 in 2024-04-17T07:50:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:50:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:46 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/1207503685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:50:47.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:46 smithi046 ceph-mon[32038]: pgmap v121: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-17T07:50:47.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:46 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/244337252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]: dispatch 2024-04-17T07:50:47.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:46 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]: dispatch 2024-04-17T07:50:47.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:46 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fb57543-bfba-422e-9306-4b1b50a98c0b"}]': finished 2024-04-17T07:50:47.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:46 smithi046 ceph-mon[32038]: osdmap e30: 5 total, 4 up, 5 in 2024-04-17T07:50:47.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:50:47.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:46 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/1207503685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:50:47.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:47 smithi053 ceph-mon[36437]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-17T07:50:47.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:47 smithi053 ceph-mon[36437]: Cluster is now healthy 2024-04-17T07:50:48.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:47 smithi062 ceph-mon[36507]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-17T07:50:48.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:47 smithi062 ceph-mon[36507]: Cluster is now healthy 2024-04-17T07:50:48.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:47 smithi046 ceph-mon[32038]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-17T07:50:48.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:47 smithi046 ceph-mon[32038]: Cluster is now healthy 2024-04-17T07:50:48.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:48 smithi053 ceph-mon[36437]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-17T07:50:49.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:48 smithi062 ceph-mon[36507]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-17T07:50:49.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:48 smithi046 ceph-mon[32038]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-17T07:50:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:50 smithi053 ceph-mon[36437]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:50:51.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:50 smithi062 ceph-mon[36507]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:50:51.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:50 smithi046 ceph-mon[32038]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:50:52.834 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:52 smithi046 ceph-mon[32038]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-17T07:50:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:52 smithi053 ceph-mon[36437]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-17T07:50:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:52 smithi062 ceph-mon[36507]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-17T07:50:54.024 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:53 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T07:50:54.024 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:53 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:54.025 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:53 smithi053 ceph-mon[36437]: Deploying daemon osd.4 on smithi053 2024-04-17T07:50:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:53 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T07:50:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:53 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:53 smithi062 ceph-mon[36507]: Deploying daemon osd.4 on smithi053 2024-04-17T07:50:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:53 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T07:50:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:53 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:50:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:53 smithi046 ceph-mon[32038]: Deploying daemon osd.4 on smithi053 2024-04-17T07:50:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:54 smithi053 ceph-mon[36437]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:50:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:54 smithi062 ceph-mon[36507]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:50:55.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:54 smithi046 ceph-mon[32038]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:50:56.942 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:56 smithi053 ceph-mon[36437]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:50:57.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:56 smithi062 ceph-mon[36507]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:50:57.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:56 smithi046 ceph-mon[32038]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:50:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:58 smithi062 ceph-mon[36507]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:50:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:50:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:58 smithi046 ceph-mon[32038]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:50:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:50:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:58 smithi053 ceph-mon[36437]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:50:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:50:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:50:59.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:50:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:00.858 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:00 smithi053 ceph-mon[36437]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:01.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:00 smithi062 ceph-mon[36507]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:00 smithi046 ceph-mon[32038]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:02.043 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.043+0000 7fb3fdffb700 1 -- 172.21.15.53:0/1064991285 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb4000ff190 con 0x7fb3f00a3c40 2024-04-17T07:51:02.044 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 4 on host 'smithi053' 2024-04-17T07:51:02.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb3f00a3c40 msgr2=0x7fb3f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb3f00a3c40 0x7fb3f00a6100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb400102a10 tx=0x7fb3f400b410 comp rx=0 tx=0).stop 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400103550 msgr2=0x7fb4001044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400103550 0x7fb4001044d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800ada0 tx=0x7fb3f800c330 comp rx=0 tx=0).stop 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 shutdown_connections 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb3f00a3c40 0x7fb3f00a6100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb400103550 0x7fb4001044d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4001019b0 0x7fb400105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 --2- 172.21.15.53:0/1064991285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb400102bb0 0x7fb400103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 >> 172.21.15.53:0/1064991285 conn(0x7fb4000fcf50 msgr2=0x7fb4000fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 shutdown_connections 2024-04-17T07:51:02.047 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:51:02.045+0000 7fb406b31700 1 -- 172.21.15.53:0/1064991285 wait complete. 2024-04-17T07:51:02.677 DEBUG:teuthology.orchestra.run.smithi053:osd.4> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.4.service 2024-04-17T07:51:02.679 INFO:tasks.cephadm:Deploying osd.5 on smithi062 with /dev/vg_nvme/lv_3... 2024-04-17T07:51:02.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T07:51:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:51:03.258 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:03 smithi062 ceph-mon[36507]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:03.258 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.258 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.258 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.258 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.259 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:03 smithi062 ceph-mon[36507]: from='osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:51:03.259 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:03 smithi062 ceph-mon[36507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:51:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:03 smithi046 ceph-mon[32038]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:03 smithi046 ceph-mon[32038]: from='osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:51:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:03 smithi046 ceph-mon[32038]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:51:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:03 smithi053 ceph-mon[36437]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:03.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:03 smithi053 ceph-mon[36437]: from='osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:51:03.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:03 smithi053 ceph-mon[36437]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:51:04.342 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:04 smithi053 ceph-mon[36437]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T07:51:04.342 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:04 smithi053 ceph-mon[36437]: from='osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:51:04.342 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:04 smithi053 ceph-mon[36437]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T07:51:04.342 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:04.342 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:04 smithi053 ceph-mon[36437]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:51:04.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:04 smithi046 ceph-mon[32038]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T07:51:04.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:04 smithi046 ceph-mon[32038]: from='osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:51:04.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:04 smithi046 ceph-mon[32038]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T07:51:04.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:04.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:04 smithi046 ceph-mon[32038]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:51:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:04 smithi062 ceph-mon[36507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T07:51:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:04 smithi062 ceph-mon[36507]: from='osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:51:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:04 smithi062 ceph-mon[36507]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T07:51:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:04 smithi062 ceph-mon[36507]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:51:04.972 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 07:51:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4[47598]: 2024-04-17T07:51:04.646+0000 7fe773abb700 -1 osd.4 0 waiting for initial osdmap 2024-04-17T07:51:04.972 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 07:51:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4[47598]: 2024-04-17T07:51:04.652+0000 7fe76b0a5700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:51:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T07:51:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T07:51:05.160 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-17T07:51:05.160 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-17T07:51:05.161 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110749 s, 947 MB/s 2024-04-17T07:51:05.161 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-17T07:51:05.334 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:05 smithi062 ceph-mon[36507]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:05.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:05 smithi062 ceph-mon[36507]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:51:05.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:05 smithi062 ceph-mon[36507]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T07:51:05.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:05.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:05 smithi062 ceph-mon[36507]: from='osd.4 ' entity='osd.4' 2024-04-17T07:51:05.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:05.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:05 smithi046 ceph-mon[32038]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:05.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:05 smithi046 ceph-mon[32038]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:51:05.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:05 smithi046 ceph-mon[32038]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T07:51:05.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:05.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:05 smithi046 ceph-mon[32038]: from='osd.4 ' entity='osd.4' 2024-04-17T07:51:05.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:05.429 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:05 smithi053 ceph-mon[36437]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:05.429 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:05 smithi053 ceph-mon[36437]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:51:05.429 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:05 smithi053 ceph-mon[36437]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T07:51:05.429 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:05.429 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:05 smithi053 ceph-mon[36437]: from='osd.4 ' entity='osd.4' 2024-04-17T07:51:05.429 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:05.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-04-17T07:51:06.087 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] boot 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: osdmap e33: 5 total, 5 up, 5 in 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:06.354 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:06.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:06.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] boot 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: osdmap e33: 5 total, 5 up, 5 in 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:06.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: osd.4 [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] boot 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: osdmap e33: 5 total, 5 up, 5 in 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:06.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:07.264 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:07 smithi062 ceph-mon[36507]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:07.264 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:07 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi053 2024-04-17T07:51:07.264 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:07 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi053 to 8542M 2024-04-17T07:51:07.264 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:07 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:07.264 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:07 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:07.264 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:07 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:07.264 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:07 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:07 smithi046 ceph-mon[32038]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:07 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi053 2024-04-17T07:51:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:07 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi053 to 8542M 2024-04-17T07:51:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:07 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:07.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:07 smithi053 ceph-mon[36437]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-17T07:51:07.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:07 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi053 2024-04-17T07:51:07.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:07 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi053 to 8542M 2024-04-17T07:51:07.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:07 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:07.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:07 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:07.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:07 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:07.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:07 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 -- 172.21.15.62:0/318356898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8102ad0 msgr2=0x7f36b8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 --2- 172.21.15.62:0/318356898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8102ad0 0x7f36b8102ef0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f36a800b0a0 tx=0x7f36a800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:51:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 -- 172.21.15.62:0/318356898 shutdown_connections 2024-04-17T07:51:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 --2- 172.21.15.62:0/318356898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b8104670 0x7f36b810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 --2- 172.21.15.62:0/318356898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8102ad0 0x7f36b8102ef0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 --2- 172.21.15.62:0/318356898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36b8103cd0 0x7f36b8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 -- 172.21.15.62:0/318356898 >> 172.21.15.62:0/318356898 conn(0x7f36b80fe070 msgr2=0x7f36b81004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 -- 172.21.15.62:0/318356898 shutdown_connections 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.765+0000 7f36bf856700 1 -- 172.21.15.62:0/318356898 wait complete. 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 Processor -- start 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 -- start start 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36b8102ad0 0x7f36b80767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b8103cd0 0x7f36b8074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8104670 0x7f36b8075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810e710 con 0x7f36b8102ad0 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810e590 con 0x7f36b8103cd0 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bf856700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36b810e890 con 0x7f36b8104670 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bddf3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8104670 0x7f36b8075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bddf3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8104670 0x7f36b8075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52484/0 (socket says 172.21.15.62:52484) 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bddf3700 1 -- 172.21.15.62:0/1949543262 learned_addr learned my addr 172.21.15.62:0/1949543262 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:51:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.766+0000 7f36bcdf1700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b8103cd0 0x7f36b8074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bddf3700 1 -- 172.21.15.62:0/1949543262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b8103cd0 msgr2=0x7f36b8074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bddf3700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b8103cd0 0x7f36b8074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bddf3700 1 -- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36b8102ad0 msgr2=0x7f36b80767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bddf3700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36b8102ad0 0x7f36b80767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bddf3700 1 -- 172.21.15.62:0/1949543262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b8075c30 con 0x7f36b8104670 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bddf3700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8104670 0x7f36b8075380 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c900 tx=0x7f36b000cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36b67fc700 1 -- 172.21.15.62:0/1949543262 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0007b00 con 0x7f36b8104670 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36b67fc700 1 -- 172.21.15.62:0/1949543262 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b000ce50 con 0x7f36b8104670 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36b67fc700 1 -- 172.21.15.62:0/1949543262 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0018660 con 0x7f36b8104670 2024-04-17T07:51:07.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b81a2e20 con 0x7f36b8104670 2024-04-17T07:51:07.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.767+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b81a3300 con 0x7f36b8104670 2024-04-17T07:51:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.768+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b8069740 con 0x7f36b8104670 2024-04-17T07:51:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.769+0000 7f36b67fc700 1 -- 172.21.15.62:0/1949543262 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36b000f450 con 0x7f36b8104670 2024-04-17T07:51:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.770+0000 7f36b67fc700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36a00a3cc0 0x7f36a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.770+0000 7f36b67fc700 1 -- 172.21.15.62:0/1949543262 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f36b00cf260 con 0x7f36b8104670 2024-04-17T07:51:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.770+0000 7f36bd5f2700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36a00a3cc0 0x7f36a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:51:07.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.771+0000 7f36bd5f2700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36a00a3cc0 0x7f36a00a6180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009be0 tx=0x7f36a8008fe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:51:07.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.775+0000 7f36b67fc700 1 -- 172.21.15.62:0/1949543262 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f36b0014030 con 0x7f36b8104670 2024-04-17T07:51:07.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:07.991+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f36b8062e90 con 0x7f36a00a3cc0 2024-04-17T07:51:08.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:08 smithi053 ceph-mon[36437]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T07:51:08.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:08 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:51:08.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:08 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:51:08.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:08 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:08.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:08 smithi062 ceph-mon[36507]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T07:51:08.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:08 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:51:08.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:08 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:51:08.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:08 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:08.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:08 smithi046 ceph-mon[32038]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T07:51:08.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:51:08.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:51:08.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:09 smithi062 ceph-mon[36507]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:09 smithi062 ceph-mon[36507]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:51:09.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:09 smithi053 ceph-mon[36437]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:09.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:09 smithi053 ceph-mon[36437]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:51:09.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:09 smithi046 ceph-mon[32038]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:09.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:09 smithi046 ceph-mon[32038]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:51:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:11 smithi053 ceph-mon[36437]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:11 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/673433848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]: dispatch 2024-04-17T07:51:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:11 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]: dispatch 2024-04-17T07:51:11.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:11 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]': finished 2024-04-17T07:51:11.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:11 smithi053 ceph-mon[36437]: osdmap e35: 6 total, 5 up, 6 in 2024-04-17T07:51:11.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:11 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:11.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:11 smithi062 ceph-mon[36507]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:11.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:11 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/673433848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]: dispatch 2024-04-17T07:51:11.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:11 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]: dispatch 2024-04-17T07:51:11.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:11 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]': finished 2024-04-17T07:51:11.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:11 smithi062 ceph-mon[36507]: osdmap e35: 6 total, 5 up, 6 in 2024-04-17T07:51:11.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:11 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:11.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:11 smithi046 ceph-mon[32038]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:11.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:11 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/673433848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]: dispatch 2024-04-17T07:51:11.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:11 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]: dispatch 2024-04-17T07:51:11.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:11 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26424544-5d91-45e4-8e62-00945c375bb5"}]': finished 2024-04-17T07:51:11.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:11 smithi046 ceph-mon[32038]: osdmap e35: 6 total, 5 up, 6 in 2024-04-17T07:51:11.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:11 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:12.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:12 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/1269734552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:51:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:12 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/1269734552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:51:12.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:12 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/1269734552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:51:13.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:13 smithi053 ceph-mon[36437]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:13 smithi062 ceph-mon[36507]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:13.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:13 smithi046 ceph-mon[32038]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:15.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:15 smithi053 ceph-mon[36437]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:15.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:15 smithi062 ceph-mon[36507]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:15.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:15 smithi046 ceph-mon[32038]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:17.298 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:17 smithi062 ceph-mon[36507]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:17.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:17 smithi053 ceph-mon[36437]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:17.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:17 smithi046 ceph-mon[32038]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:19.373 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:19 smithi062 ceph-mon[36507]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:19.373 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T07:51:19.373 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:19 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:19.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:19 smithi053 ceph-mon[36437]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:19.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T07:51:19.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:19 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:19.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:19 smithi046 ceph-mon[32038]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:19.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T07:51:19.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:19 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:20.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:20 smithi053 ceph-mon[36437]: Deploying daemon osd.5 on smithi062 2024-04-17T07:51:20.512 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:20 smithi062 ceph-mon[36507]: Deploying daemon osd.5 on smithi062 2024-04-17T07:51:20.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:20 smithi046 ceph-mon[32038]: Deploying daemon osd.5 on smithi062 2024-04-17T07:51:21.512 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:21 smithi046 ceph-mon[32038]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:21.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:21 smithi062 ceph-mon[36507]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:21.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:21 smithi053 ceph-mon[36437]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:22.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:22 smithi053 ceph-mon[36437]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:22.498 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:22 smithi062 ceph-mon[36507]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:22.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:22 smithi046 ceph-mon[32038]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:24.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:24 smithi062 ceph-mon[36507]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:24.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:24 smithi046 ceph-mon[32038]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:24.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:24 smithi053 ceph-mon[36437]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:25.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:25.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:25.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:26 smithi062 ceph-mon[36507]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:26.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:26 smithi046 ceph-mon[32038]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:26.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:26 smithi053 ceph-mon[36437]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:28.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.027+0000 7f36b67fc700 1 -- 172.21.15.62:0/1949543262 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f36b8062e90 con 0x7f36a00a3cc0 2024-04-17T07:51:28.027 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 5 on host 'smithi062' 2024-04-17T07:51:28.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36a00a3cc0 msgr2=0x7f36a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:28.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36a00a3cc0 0x7f36a00a6180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009be0 tx=0x7f36a8008fe0 comp rx=0 tx=0).stop 2024-04-17T07:51:28.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8104670 msgr2=0x7f36b8075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8104670 0x7f36b8075380 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c900 tx=0x7f36b000cc10 comp rx=0 tx=0).stop 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 shutdown_connections 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36a00a3cc0 0x7f36a00a6180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36b8104670 0x7f36b8075380 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36b8102ad0 0x7f36b80767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 --2- 172.21.15.62:0/1949543262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b8103cd0 0x7f36b8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 >> 172.21.15.62:0/1949543262 conn(0x7f36b80fe070 msgr2=0x7f36b8106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 shutdown_connections 2024-04-17T07:51:28.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:51:28.030+0000 7f36bf856700 1 -- 172.21.15.62:0/1949543262 wait complete. 2024-04-17T07:51:28.702 DEBUG:teuthology.orchestra.run.smithi062:osd.5> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.5.service 2024-04-17T07:51:28.705 INFO:tasks.cephadm:Deploying osd.6 on smithi046 with /dev/vg_nvme/lv_2... 2024-04-17T07:51:28.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T07:51:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:29 smithi046 ceph-mon[32038]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:29 smithi046 ceph-mon[32038]: from='osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:51:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:29 smithi046 ceph-mon[32038]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:51:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:51:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:29 smithi062 ceph-mon[36507]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:29 smithi062 ceph-mon[36507]: from='osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:51:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:29 smithi062 ceph-mon[36507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:51:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:29 smithi053 ceph-mon[36437]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:29 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:29 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:29 smithi053 ceph-mon[36437]: from='osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:51:29.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:29 smithi053 ceph-mon[36437]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:51:29.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:29 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:29.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:29 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:30.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:30 smithi046 ceph-mon[32038]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T07:51:30.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:30 smithi046 ceph-mon[32038]: from='osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:51:30.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:30 smithi046 ceph-mon[32038]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T07:51:30.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:30.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:30 smithi046 ceph-mon[32038]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:51:30.381 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:30 smithi062 ceph-mon[36507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T07:51:30.381 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:30 smithi062 ceph-mon[36507]: from='osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:51:30.381 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:30 smithi062 ceph-mon[36507]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T07:51:30.381 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:30.381 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:30 smithi062 ceph-mon[36507]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:51:30.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:30 smithi053 ceph-mon[36437]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T07:51:30.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:30 smithi053 ceph-mon[36437]: from='osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:51:30.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:30 smithi053 ceph-mon[36437]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T07:51:30.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:30 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:30.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:30 smithi053 ceph-mon[36437]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:51:31.053 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 07:51:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T07:51:30.721+0000 7fd09c70d700 -1 osd.5 0 waiting for initial osdmap 2024-04-17T07:51:31.054 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 07:51:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T07:51:30.729+0000 7fd0944f8700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:51:31.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:31.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:51:31.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T07:51:31.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:31.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: from='osd.5 ' entity='osd.5' 2024-04-17T07:51:31.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] boot 2024-04-17T07:51:31.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: osdmap e38: 6 total, 6 up, 6 in 2024-04-17T07:51:31.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:51:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T07:51:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: from='osd.5 ' entity='osd.5' 2024-04-17T07:51:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] boot 2024-04-17T07:51:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: osdmap e38: 6 total, 6 up, 6 in 2024-04-17T07:51:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: from='osd.5 ' entity='osd.5' 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: osd.5 [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] boot 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: osdmap e38: 6 total, 6 up, 6 in 2024-04-17T07:51:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:51:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T07:51:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T07:51:32.060 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-17T07:51:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-17T07:51:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0385202 s, 272 MB/s 2024-04-17T07:51:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-17T07:51:32.281 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:32 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:51:32.281 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:32 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:51:32.281 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:32 smithi062 ceph-mon[36507]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T07:51:32.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:32 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:51:32.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:32 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:51:32.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:32 smithi046 ceph-mon[32038]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T07:51:32.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:32 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:51:32.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:32 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:51:32.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:32 smithi053 ceph-mon[36437]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T07:51:32.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-04-17T07:51:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:51:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:33 smithi046 ceph-mon[32038]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:33.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:33.473 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:33.473 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.473 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:33 smithi053 ceph-mon[36437]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T07:51:33.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:33.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:33.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:33 smithi062 ceph-mon[36507]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T07:51:34.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:34 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi062 2024-04-17T07:51:34.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:34 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi062 to 8540M 2024-04-17T07:51:34.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:34 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi062 2024-04-17T07:51:34.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:34 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi062 to 8540M 2024-04-17T07:51:34.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:34 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi062 2024-04-17T07:51:34.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:34 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi062 to 8540M 2024-04-17T07:51:35.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:35 smithi046 ceph-mon[32038]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:35.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:35 smithi046 ceph-mon[32038]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T07:51:35.454 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:35 smithi062 ceph-mon[36507]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:35.454 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:35 smithi062 ceph-mon[36507]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T07:51:35.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:35 smithi053 ceph-mon[36437]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:35 smithi053 ceph-mon[36437]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T07:51:35.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 -- 172.21.15.46:0/3125147099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 msgr2=0x7f870c10a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 --2- 172.21.15.46:0/3125147099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c10a750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009990 tx=0x7f86f8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 -- 172.21.15.46:0/3125147099 shutdown_connections 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 --2- 172.21.15.46:0/3125147099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c10ac90 0x7f870c111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 --2- 172.21.15.46:0/3125147099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f870c1090f0 0x7f870c109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 --2- 172.21.15.46:0/3125147099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c10a750 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 -- 172.21.15.46:0/3125147099 >> 172.21.15.46:0/3125147099 conn(0x7f870c0780c0 msgr2=0x7f870c07a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 -- 172.21.15.46:0/3125147099 shutdown_connections 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.632+0000 7f8712200700 1 -- 172.21.15.46:0/3125147099 wait complete. 2024-04-17T07:51:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 Processor -- start 2024-04-17T07:51:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 -- start start 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c1090f0 0x7f870c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f870c10ac90 0x7f870c1a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f870c114d30 con 0x7f870c1090f0 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f870c114bb0 con 0x7f870c10ac90 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f8712200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f870c114eb0 con 0x7f870c10a2f0 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f870aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f870b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c1090f0 0x7f870c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:51:35.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f870aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39740/0 (socket says 172.21.15.46:39740) 2024-04-17T07:51:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f870aa00700 1 -- 172.21.15.46:0/1196056848 learned_addr learned my addr 172.21.15.46:0/1196056848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:51:35.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.635+0000 7f870be00700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f870c10ac90 0x7f870c1a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870aa00700 1 -- 172.21.15.46:0/1196056848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f870c10ac90 msgr2=0x7f870c1a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870aa00700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f870c10ac90 0x7f870c1a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870aa00700 1 -- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c1090f0 msgr2=0x7f870c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870aa00700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c1090f0 0x7f870c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870aa00700 1 -- 172.21.15.46:0/1196056848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c1a6f20 con 0x7f870c10a2f0 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870b400700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c1090f0 0x7f870c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870be00700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f870c10ac90 0x7f870c1a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f870aa00700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c19f800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f86f800f690 tx=0x7f86f800f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f8703e00700 1 -- 172.21.15.46:0/1196056848 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8026070 con 0x7f870c10a2f0 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f8703e00700 1 -- 172.21.15.46:0/1196056848 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f80218f0 con 0x7f870c10a2f0 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f8703e00700 1 -- 172.21.15.46:0/1196056848 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8017650 con 0x7f870c10a2f0 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.636+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c1a71b0 con 0x7f870c10a2f0 2024-04-17T07:51:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.637+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a7660 con 0x7f870c10a2f0 2024-04-17T07:51:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.638+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870c069740 con 0x7f870c10a2f0 2024-04-17T07:51:35.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.640+0000 7f8703e00700 1 -- 172.21.15.46:0/1196056848 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f86f8029070 con 0x7f870c10a2f0 2024-04-17T07:51:35.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.641+0000 7f8703e00700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f86f00a3cc0 0x7f86f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:51:35.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.641+0000 7f8703e00700 1 -- 172.21.15.46:0/1196056848 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f86f80d0330 con 0x7f870c10a2f0 2024-04-17T07:51:35.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.642+0000 7f870b400700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f86f00a3cc0 0x7f86f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:51:35.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.642+0000 7f870b400700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f86f00a3cc0 0x7f86f00a6180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f870c10a150 tx=0x7f86fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:51:35.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.646+0000 7f8703e00700 1 -- 172.21.15.46:0/1196056848 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f86f8013070 con 0x7f870c10a2f0 2024-04-17T07:51:35.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:51:35.889+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f870c062f50 con 0x7f86f00a3cc0 2024-04-17T07:51:36.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:36 smithi046 ceph-mon[32038]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T07:51:36.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:51:36.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:51:36.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:36 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:36.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:36 smithi053 ceph-mon[36437]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T07:51:36.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:51:36.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:51:36.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:36 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:36.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:36 smithi062 ceph-mon[36507]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T07:51:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:51:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:51:36.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:36 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:37.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:37 smithi046 ceph-mon[32038]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:37.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:37 smithi046 ceph-mon[32038]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:51:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:37 smithi053 ceph-mon[36437]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:37.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:37 smithi053 ceph-mon[36437]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:51:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:37 smithi062 ceph-mon[36507]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 502 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:37 smithi062 ceph-mon[36507]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:51:39.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:39 smithi046 ceph-mon[32038]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:51:39.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:39 smithi053 ceph-mon[36437]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:51:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:39 smithi062 ceph-mon[36507]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:51:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:40 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2871435958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "720d8bdb-9a6a-4fcc-bda7-b575a895ea17"}]: dispatch 2024-04-17T07:51:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:40 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2871435958' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "720d8bdb-9a6a-4fcc-bda7-b575a895ea17"}]': finished 2024-04-17T07:51:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:40 smithi053 ceph-mon[36437]: osdmap e43: 7 total, 6 up, 7 in 2024-04-17T07:51:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:51:40.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:40 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2871435958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "720d8bdb-9a6a-4fcc-bda7-b575a895ea17"}]: dispatch 2024-04-17T07:51:40.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:40 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2871435958' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "720d8bdb-9a6a-4fcc-bda7-b575a895ea17"}]': finished 2024-04-17T07:51:40.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:40 smithi062 ceph-mon[36507]: osdmap e43: 7 total, 6 up, 7 in 2024-04-17T07:51:40.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:51:40.573 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:40 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2871435958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "720d8bdb-9a6a-4fcc-bda7-b575a895ea17"}]: dispatch 2024-04-17T07:51:40.573 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:40 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2871435958' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "720d8bdb-9a6a-4fcc-bda7-b575a895ea17"}]': finished 2024-04-17T07:51:40.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:40 smithi046 ceph-mon[32038]: osdmap e43: 7 total, 6 up, 7 in 2024-04-17T07:51:40.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:51:41.373 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:41 smithi053 ceph-mon[36437]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:51:41.373 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:41 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2617283003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:51:41.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:41 smithi062 ceph-mon[36507]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:51:41.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:41 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2617283003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:51:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:41 smithi046 ceph-mon[32038]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:51:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:41 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2617283003' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:51:43.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:43 smithi053 ceph-mon[36437]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-17T07:51:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:43 smithi062 ceph-mon[36507]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-17T07:51:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:43 smithi046 ceph-mon[32038]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-17T07:51:45.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:45 smithi053 ceph-mon[36437]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:51:45.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:45 smithi062 ceph-mon[36507]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:51:45.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:45 smithi046 ceph-mon[32038]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:51:47.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:47 smithi046 ceph-mon[32038]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:51:47.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:47 smithi053 ceph-mon[36437]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:51:47.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:47 smithi062 ceph-mon[36507]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:51:49.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:49 smithi046 ceph-mon[32038]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:49.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:49 smithi053 ceph-mon[36437]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:49.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:49 smithi062 ceph-mon[36507]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:50.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:50 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T07:51:50.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:50 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:50.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:50 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T07:51:50.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:50 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:50.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:50 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T07:51:50.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:50 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:51:51.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:51 smithi053 ceph-mon[36437]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:51.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:51 smithi053 ceph-mon[36437]: Deploying daemon osd.6 on smithi046 2024-04-17T07:51:51.480 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:51 smithi046 ceph-mon[32038]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:51.481 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:51 smithi046 ceph-mon[32038]: Deploying daemon osd.6 on smithi046 2024-04-17T07:51:51.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:51 smithi062 ceph-mon[36507]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:51.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:51 smithi062 ceph-mon[36507]: Deploying daemon osd.6 on smithi046 2024-04-17T07:51:53.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:53 smithi046 ceph-mon[32038]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:53.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:53 smithi053 ceph-mon[36437]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:53.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:53 smithi062 ceph-mon[36507]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:55.322 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:55 smithi046 ceph-mon[32038]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:55.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:55 smithi053 ceph-mon[36437]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:55.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:55 smithi062 ceph-mon[36507]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:57.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:57 smithi053 ceph-mon[36437]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:57 smithi062 ceph-mon[36507]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:57.592 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:57 smithi046 ceph-mon[32038]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:59 smithi053 ceph-mon[36437]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:59.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:59.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:51:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:59.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:59 smithi046 ceph-mon[32038]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:59.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:59.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:59.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:51:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:59 smithi062 ceph-mon[36507]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:51:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:51:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:51:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:51:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:01.350 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:01 smithi046 ceph-mon[32038]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:01.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:01 smithi053 ceph-mon[36437]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:01.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:01 smithi062 ceph-mon[36507]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:03.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:02 smithi046 ceph-mon[32038]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:03.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:03.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:03.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:02 smithi046 ceph-mon[32038]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T07:52:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:02 smithi053 ceph-mon[36437]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:02 smithi053 ceph-mon[36437]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T07:52:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:02 smithi062 ceph-mon[36507]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:02 smithi062 ceph-mon[36507]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T07:52:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.840+0000 7f8703e00700 1 -- 172.21.15.46:0/1196056848 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f870c062f50 con 0x7f86f00a3cc0 2024-04-17T07:52:03.841 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 6 on host 'smithi046' 2024-04-17T07:52:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f86f00a3cc0 msgr2=0x7f86f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f86f00a3cc0 0x7f86f00a6180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f870c10a150 tx=0x7f86fc009260 comp rx=0 tx=0).stop 2024-04-17T07:52:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 msgr2=0x7f870c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c19f800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f86f800f690 tx=0x7f86f800f6c0 comp rx=0 tx=0).stop 2024-04-17T07:52:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 shutdown_connections 2024-04-17T07:52:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f86f00a3cc0 0x7f86f00a6180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f870c10ac90 0x7f870c1a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:03.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c1090f0 0x7f870c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:03.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 --2- 172.21.15.46:0/1196056848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f870c10a2f0 0x7f870c19f800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:03.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.844+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 >> 172.21.15.46:0/1196056848 conn(0x7f870c0780c0 msgr2=0x7f870c10d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:52:03.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.845+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 shutdown_connections 2024-04-17T07:52:03.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:52:03.845+0000 7f8712200700 1 -- 172.21.15.46:0/1196056848 wait complete. 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:04.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T07:52:04.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T07:52:04.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:52:04.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:04.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:04.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:04.729 DEBUG:teuthology.orchestra.run.smithi046:osd.6> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.6.service 2024-04-17T07:52:04.734 INFO:tasks.cephadm:Deploying osd.7 on smithi053 with /dev/vg_nvme/lv_2... 2024-04-17T07:52:04.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T07:52:04.993 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:52:05.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:05 smithi046 ceph-mon[32038]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:05.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:05 smithi046 ceph-mon[32038]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:52:05.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:05 smithi046 ceph-mon[32038]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T07:52:05.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:05.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:05.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:05 smithi053 ceph-mon[36437]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:05.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:05 smithi053 ceph-mon[36437]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:52:05.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:05 smithi053 ceph-mon[36437]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T07:52:05.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:05.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:05 smithi062 ceph-mon[36507]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:05 smithi062 ceph-mon[36507]: from='osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:52:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:05 smithi062 ceph-mon[36507]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T07:52:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:05.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:06.396 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:06 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:52:06.396 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:06 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:52:06.396 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:06 smithi053 ceph-mon[36437]: osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] boot 2024-04-17T07:52:06.396 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:06 smithi053 ceph-mon[36437]: osdmap e46: 7 total, 7 up, 7 in 2024-04-17T07:52:06.396 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:06.460 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:06 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:52:06.460 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:06 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:52:06.460 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:06 smithi046 ceph-mon[32038]: osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] boot 2024-04-17T07:52:06.460 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:06 smithi046 ceph-mon[32038]: osdmap e46: 7 total, 7 up, 7 in 2024-04-17T07:52:06.460 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:06.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:06 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:52:06.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:06 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:52:06.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:06 smithi062 ceph-mon[36507]: osd.6 [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] boot 2024-04-17T07:52:06.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:06 smithi062 ceph-mon[36507]: osdmap e46: 7 total, 7 up, 7 in 2024-04-17T07:52:06.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:52:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T07:52:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T07:52:06.971 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-17T07:52:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-17T07:52:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00806655 s, 1.3 GB/s 2024-04-17T07:52:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-17T07:52:07.284 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:07 smithi053 ceph-mon[36437]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:07.285 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:07 smithi053 ceph-mon[36437]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T07:52:07.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:07 smithi062 ceph-mon[36507]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:07.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:07 smithi062 ceph-mon[36507]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T07:52:07.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:07 smithi046 ceph-mon[32038]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:52:07.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:07 smithi046 ceph-mon[32038]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T07:52:07.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-04-17T07:52:08.038 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:52:09.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:09 smithi053 ceph-mon[36437]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:09 smithi062 ceph-mon[36507]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:09.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:09 smithi046 ceph-mon[32038]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 -- 172.21.15.53:0/3321832669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0581040c0 msgr2=0x7ff0581044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 --2- 172.21.15.53:0/3321832669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0581040c0 0x7ff0581044e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0540099c0 tx=0x7ff054009cd0 comp rx=0 tx=0).stop 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 -- 172.21.15.53:0/3321832669 shutdown_connections 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 --2- 172.21.15.53:0/3321832669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff058101500 0x7ff05810b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 --2- 172.21.15.53:0/3321832669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0581040c0 0x7ff0581044e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 --2- 172.21.15.53:0/3321832669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff058100b40 0x7ff058100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 -- 172.21.15.53:0/3321832669 >> 172.21.15.53:0/3321832669 conn(0x7ff0580fc6c0 msgr2=0x7ff0580feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:52:09.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 -- 172.21.15.53:0/3321832669 shutdown_connections 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 -- 172.21.15.53:0/3321832669 wait complete. 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.867+0000 7ff060455700 1 Processor -- start 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff060455700 1 -- start start 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff060455700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff058100b40 0x7ff0581037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff060455700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff058101500 0x7ff058101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff060455700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0581040c0 0x7ff058102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff060455700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff05810e760 con 0x7ff0581040c0 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff060455700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff05810e5e0 con 0x7ff058101500 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff060455700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff05810e8e0 con 0x7ff058100b40 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff058101500 0x7ff058101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff058101500 0x7ff058101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52830/0 (socket says 172.21.15.53:52830) 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 -- 172.21.15.53:0/1870278304 learned_addr learned my addr 172.21.15.53:0/1870278304 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-17T07:52:09.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05e1f1700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff058100b40 0x7ff0581037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05e9f2700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0581040c0 0x7ff058102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 -- 172.21.15.53:0/1870278304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff058100b40 msgr2=0x7ff0581037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff058100b40 0x7ff0581037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 -- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0581040c0 msgr2=0x7ff058102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0581040c0 0x7ff058102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 -- 172.21.15.53:0/1870278304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff058102c00 con 0x7ff058101500 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05e9f2700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0581040c0 0x7ff058102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.868+0000 7ff05d9f0700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff058101500 0x7ff058101e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff04800b500 tx=0x7ff04800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.869+0000 7ff04f7fe700 1 -- 172.21.15.53:0/1870278304 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048015020 con 0x7ff058101500 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.869+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0581a2ee0 con 0x7ff058101500 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.869+0000 7ff04f7fe700 1 -- 172.21.15.53:0/1870278304 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff048011d10 con 0x7ff058101500 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.869+0000 7ff04f7fe700 1 -- 172.21.15.53:0/1870278304 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048010e50 con 0x7ff058101500 2024-04-17T07:52:09.871 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.869+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0581a32e0 con 0x7ff058101500 2024-04-17T07:52:09.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.870+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05802df90 con 0x7ff058101500 2024-04-17T07:52:09.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.871+0000 7ff04f7fe700 1 -- 172.21.15.53:0/1870278304 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff048018020 con 0x7ff058101500 2024-04-17T07:52:09.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.871+0000 7ff04f7fe700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0440a3de0 0x7ff0440a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:09.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.872+0000 7ff04f7fe700 1 -- 172.21.15.53:0/1870278304 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7ff0480cee40 con 0x7ff058101500 2024-04-17T07:52:09.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.872+0000 7ff05e1f1700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0440a3de0 0x7ff0440a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:52:09.873 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.872+0000 7ff05e1f1700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0440a3de0 0x7ff0440a62a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff054009990 tx=0x7ff054005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:52:09.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:09.876+0000 7ff04f7fe700 1 -- 172.21.15.53:0/1870278304 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff04801f020 con 0x7ff058101500 2024-04-17T07:52:10.094 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:10.094+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff058062e90 con 0x7ff0440a3de0 2024-04-17T07:52:10.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:10.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi046 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi046 to 5698M 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:52:10.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:10 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi046 2024-04-17T07:52:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi046 to 5698M 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:52:11.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:10 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi046 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi046 to 5698M 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:11.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:52:11.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:52:11.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:52:11.112 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:10 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:12.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:12 smithi053 ceph-mon[36437]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:13.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:12 smithi062 ceph-mon[36507]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:13.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:12 smithi046 ceph-mon[32038]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:13.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:13 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/2094096480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]: dispatch 2024-04-17T07:52:13.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:13 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]: dispatch 2024-04-17T07:52:13.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:13 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]': finished 2024-04-17T07:52:13.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:13 smithi053 ceph-mon[36437]: osdmap e48: 8 total, 7 up, 8 in 2024-04-17T07:52:13.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:13.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:13 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/4286496741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:52:14.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:13 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/2094096480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]: dispatch 2024-04-17T07:52:14.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:13 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]: dispatch 2024-04-17T07:52:14.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:13 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]': finished 2024-04-17T07:52:14.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:13 smithi062 ceph-mon[36507]: osdmap e48: 8 total, 7 up, 8 in 2024-04-17T07:52:14.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:14.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:13 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/4286496741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:52:14.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:13 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/2094096480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]: dispatch 2024-04-17T07:52:14.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:13 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]: dispatch 2024-04-17T07:52:14.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:13 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b7d7a00-52ea-47ab-b844-d5a192861516"}]': finished 2024-04-17T07:52:14.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:13 smithi046 ceph-mon[32038]: osdmap e48: 8 total, 7 up, 8 in 2024-04-17T07:52:14.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:14.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:13 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/4286496741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:52:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:14 smithi053 ceph-mon[36437]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:15.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:14 smithi062 ceph-mon[36507]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:14 smithi046 ceph-mon[32038]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:17.170 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:16 smithi053 ceph-mon[36437]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:17.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:16 smithi062 ceph-mon[36507]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:17.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:16 smithi046 ceph-mon[32038]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:19.207 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:18 smithi053 ceph-mon[36437]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:18 smithi062 ceph-mon[36507]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:19.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:18 smithi046 ceph-mon[32038]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:21.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:20 smithi053 ceph-mon[36437]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:21.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:20 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T07:52:21.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:20 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:20 smithi062 ceph-mon[36507]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:20 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T07:52:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:20 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:21.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:20 smithi046 ceph-mon[32038]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:21.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T07:52:21.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:20 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:22.195 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:21 smithi053 ceph-mon[36437]: Deploying daemon osd.7 on smithi053 2024-04-17T07:52:22.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:21 smithi062 ceph-mon[36507]: Deploying daemon osd.7 on smithi053 2024-04-17T07:52:22.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:21 smithi046 ceph-mon[32038]: Deploying daemon osd.7 on smithi053 2024-04-17T07:52:23.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:22 smithi053 ceph-mon[36437]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:22 smithi062 ceph-mon[36507]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:23.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:22 smithi046 ceph-mon[32038]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:25.108 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:24 smithi053 ceph-mon[36437]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:24 smithi062 ceph-mon[36507]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:25.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:24 smithi046 ceph-mon[32038]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:27.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:26 smithi062 ceph-mon[36507]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:27.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:27.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:27.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:27.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:26 smithi046 ceph-mon[32038]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:27.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:27.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:27.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:26 smithi053 ceph-mon[36437]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:27.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:29.165 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:29 smithi053 ceph-mon[36437]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:29 smithi062 ceph-mon[36507]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:29.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:29 smithi046 ceph-mon[32038]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:29.532 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.531+0000 7ff04f7fe700 1 -- 172.21.15.53:0/1870278304 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff058062e90 con 0x7ff0440a3de0 2024-04-17T07:52:29.532 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 7 on host 'smithi053' 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.533+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0440a3de0 msgr2=0x7ff0440a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.533+0000 7ff060455700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0440a3de0 0x7ff0440a62a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff054009990 tx=0x7ff054005fb0 comp rx=0 tx=0).stop 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff058101500 msgr2=0x7ff058101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff058101500 0x7ff058101e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff04800b500 tx=0x7ff04800b810 comp rx=0 tx=0).stop 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 shutdown_connections 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0440a3de0 0x7ff0440a62a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0581040c0 0x7ff058102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff058100b40 0x7ff0581037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 --2- 172.21.15.53:0/1870278304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff058101500 0x7ff058101e10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 >> 172.21.15.53:0/1870278304 conn(0x7ff0580fc6c0 msgr2=0x7ff058107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:52:29.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 shutdown_connections 2024-04-17T07:52:29.535 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:52:29.534+0000 7ff060455700 1 -- 172.21.15.53:0/1870278304 wait complete. 2024-04-17T07:52:30.179 DEBUG:teuthology.orchestra.run.smithi053:osd.7> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.7.service 2024-04-17T07:52:30.181 INFO:tasks.cephadm:Deploying osd.8 on smithi062 with /dev/vg_nvme/lv_2... 2024-04-17T07:52:30.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T07:52:30.442 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:52:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:30 smithi062 ceph-mon[36507]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:30 smithi062 ceph-mon[36507]: from='osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:52:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:30 smithi062 ceph-mon[36507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:52:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:30 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:30 smithi046 ceph-mon[32038]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:30 smithi046 ceph-mon[32038]: from='osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:52:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:30 smithi046 ceph-mon[32038]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:52:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:30 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:30 smithi053 ceph-mon[36437]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:30.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:30 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:30 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:30 smithi053 ceph-mon[36437]: from='osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:52:30.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:30 smithi053 ceph-mon[36437]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:52:30.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:30 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:30.888 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:30 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:31 smithi062 ceph-mon[36507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T07:52:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:31 smithi062 ceph-mon[36507]: from='osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:52:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:31 smithi062 ceph-mon[36507]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T07:52:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:31 smithi062 ceph-mon[36507]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:52:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:31 smithi046 ceph-mon[32038]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T07:52:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:31 smithi046 ceph-mon[32038]: from='osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:52:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:31 smithi046 ceph-mon[32038]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T07:52:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:31.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:31 smithi046 ceph-mon[32038]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:52:31.891 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:31 smithi053 ceph-mon[36437]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T07:52:31.891 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:31 smithi053 ceph-mon[36437]: from='osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:52:31.891 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:31 smithi053 ceph-mon[36437]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T07:52:31.891 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:31.891 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:31 smithi053 ceph-mon[36437]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:52:31.892 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 07:52:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T07:52:31.509+0000 7f7eb59fc700 -1 osd.7 0 waiting for initial osdmap 2024-04-17T07:52:31.892 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 07:52:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T07:52:31.520+0000 7f7eb17ef700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:52:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T07:52:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T07:52:32.466 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-17T07:52:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-17T07:52:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111125 s, 944 MB/s 2024-04-17T07:52:32.467 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 2024-04-17T07:52:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-17T07:52:32.583 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:32 smithi062 ceph-mon[36507]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:32.583 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:32 smithi062 ceph-mon[36507]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:52:32.583 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:32 smithi062 ceph-mon[36507]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T07:52:32.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:32 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:32.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:32 smithi062 ceph-mon[36507]: osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] boot 2024-04-17T07:52:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:32 smithi046 ceph-mon[32038]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:32 smithi046 ceph-mon[32038]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:52:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:32 smithi046 ceph-mon[32038]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T07:52:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:32 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:32 smithi046 ceph-mon[32038]: osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] boot 2024-04-17T07:52:32.932 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:32 smithi053 ceph-mon[36437]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:52:32.932 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:32 smithi053 ceph-mon[36437]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:52:32.932 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:32 smithi053 ceph-mon[36437]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T07:52:32.932 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:32 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:32.932 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:32 smithi053 ceph-mon[36437]: osd.7 [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] boot 2024-04-17T07:52:33.242 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-04-17T07:52:33.502 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:52:33.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:33 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:52:33.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:33 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:52:33.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:33 smithi053 ceph-mon[36437]: osdmap e51: 8 total, 8 up, 8 in 2024-04-17T07:52:33.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:33.780 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:33 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:52:33.780 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:33 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:52:33.781 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:33 smithi062 ceph-mon[36507]: osdmap e51: 8 total, 8 up, 8 in 2024-04-17T07:52:33.781 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:33.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:33 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:52:33.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:33 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:52:33.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:33 smithi046 ceph-mon[32038]: osdmap e51: 8 total, 8 up, 8 in 2024-04-17T07:52:33.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:52:34.650 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:34.650 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T07:52:34.650 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi053 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi053 to 5694M 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:34.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:34 smithi062 ceph-mon[36507]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi053 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi053 to 5694M 2024-04-17T07:52:34.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:34 smithi046 ceph-mon[32038]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T07:52:34.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi053 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi053 to 5694M 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:34.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:34.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:34.973 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:34 smithi053 ceph-mon[36437]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 -- 172.21.15.62:0/3101886434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750103530 msgr2=0x7f7750105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 --2- 172.21.15.62:0/3101886434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750103530 0x7f7750105920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f774800b0a0 tx=0x7f774800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 -- 172.21.15.62:0/3101886434 shutdown_connections 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 --2- 172.21.15.62:0/3101886434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7750108790 0x7f775010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 --2- 172.21.15.62:0/3101886434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750103530 0x7f7750105920 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 --2- 172.21.15.62:0/3101886434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750105e60 0x7f7750108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 -- 172.21.15.62:0/3101886434 >> 172.21.15.62:0/3101886434 conn(0x7f77500fcf40 msgr2=0x7f77500ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:52:35.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 -- 172.21.15.62:0/3101886434 shutdown_connections 2024-04-17T07:52:35.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 -- 172.21.15.62:0/3101886434 wait complete. 2024-04-17T07:52:35.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.184+0000 7f7755ec6700 1 Processor -- start 2024-04-17T07:52:35.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f7755ec6700 1 -- start start 2024-04-17T07:52:35.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f7755ec6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7750103530 0x7f7750198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f7755ec6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750105e60 0x7f7750199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f7755ec6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750108790 0x7f77501a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f7755ec6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f775010e770 con 0x7f7750103530 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f7755ec6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f775010e5f0 con 0x7f7750108790 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f7755ec6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f775010e8f0 con 0x7f7750105e60 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750105e60 0x7f7750199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750105e60 0x7f7750199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34502/0 (socket says 172.21.15.62:34502) 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 -- 172.21.15.62:0/1758616132 learned_addr learned my addr 172.21.15.62:0/1758616132 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774ffff700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750108790 0x7f77501a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 -- 172.21.15.62:0/1758616132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750108790 msgr2=0x7f77501a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750108790 0x7f77501a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:35.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 -- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7750103530 msgr2=0x7f7750198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7750103530 0x7f7750198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.185+0000 7f774effd700 1 -- 172.21.15.62:0/1758616132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77501a09b0 con 0x7f7750105e60 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.186+0000 7f774effd700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750105e60 0x7f7750199290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f773800b560 tx=0x7f773800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.186+0000 7f774cff9700 1 -- 172.21.15.62:0/1758616132 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738015020 con 0x7f7750105e60 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.186+0000 7f774cff9700 1 -- 172.21.15.62:0/1758616132 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7738011d10 con 0x7f7750105e60 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.186+0000 7f774cff9700 1 -- 172.21.15.62:0/1758616132 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773800f2e0 con 0x7f7750105e60 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.186+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77501a0ca0 con 0x7f7750105e60 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.186+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77501a1190 con 0x7f7750105e60 2024-04-17T07:52:35.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.187+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7750069740 con 0x7f7750105e60 2024-04-17T07:52:35.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.188+0000 7f774cff9700 1 -- 172.21.15.62:0/1758616132 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7738018020 con 0x7f7750105e60 2024-04-17T07:52:35.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.189+0000 7f774cff9700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f773c0a3c40 0x7f773c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:52:35.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.189+0000 7f774cff9700 1 -- 172.21.15.62:0/1758616132 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f77380cf530 con 0x7f7750105e60 2024-04-17T07:52:35.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.189+0000 7f774f7fe700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f773c0a3c40 0x7f773c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:52:35.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.190+0000 7f774f7fe700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f773c0a3c40 0x7f773c0a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f774800b070 tx=0x7f77480060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:52:35.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.192+0000 7f774cff9700 1 -- 172.21.15.62:0/1758616132 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f773801f020 con 0x7f7750105e60 2024-04-17T07:52:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:35.413+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7750062e90 con 0x7f773c0a3c40 2024-04-17T07:52:35.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:35 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:52:35.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:35 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:52:35.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:35 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:35.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:35 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:52:35.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:35 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:52:35.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:35 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:35.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:35 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:52:35.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:35 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:52:35.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:35 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:36.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:36 smithi062 ceph-mon[36507]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:52:36.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:36 smithi062 ceph-mon[36507]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:36.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:36 smithi062 ceph-mon[36507]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T07:52:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:36 smithi046 ceph-mon[32038]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:52:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:36 smithi046 ceph-mon[32038]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:36.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:36 smithi046 ceph-mon[32038]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T07:52:36.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:36 smithi053 ceph-mon[36437]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:52:36.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:36 smithi053 ceph-mon[36437]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:36.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:36 smithi053 ceph-mon[36437]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T07:52:37.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:37 smithi062 ceph-mon[36507]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T07:52:37.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:37 smithi046 ceph-mon[32038]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T07:52:37.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:37 smithi053 ceph-mon[36437]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T07:52:38.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:38 smithi046 ceph-mon[32038]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:38.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:38 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/1527191346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]: dispatch 2024-04-17T07:52:38.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:38 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]: dispatch 2024-04-17T07:52:38.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:38 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]': finished 2024-04-17T07:52:38.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:38 smithi046 ceph-mon[32038]: osdmap e56: 9 total, 8 up, 9 in 2024-04-17T07:52:38.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:38 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:38.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:38 smithi053 ceph-mon[36437]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:38.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:38 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/1527191346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]: dispatch 2024-04-17T07:52:38.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:38 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]: dispatch 2024-04-17T07:52:38.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:38 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]': finished 2024-04-17T07:52:38.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:38 smithi053 ceph-mon[36437]: osdmap e56: 9 total, 8 up, 9 in 2024-04-17T07:52:38.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:38 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:39.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:38 smithi062 ceph-mon[36507]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:39.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:38 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/1527191346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]: dispatch 2024-04-17T07:52:39.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:38 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]: dispatch 2024-04-17T07:52:39.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:38 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f936f20-269a-496b-af26-a962f09c26a3"}]': finished 2024-04-17T07:52:39.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:38 smithi062 ceph-mon[36507]: osdmap e56: 9 total, 8 up, 9 in 2024-04-17T07:52:39.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:38 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:39.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:39 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/929002634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:52:39.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:39 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/929002634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:52:40.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:39 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/929002634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:52:40.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:40 smithi046 ceph-mon[32038]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:40.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:40 smithi053 ceph-mon[36437]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:41.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:40 smithi062 ceph-mon[36507]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:42.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:42 smithi046 ceph-mon[32038]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:42.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:42 smithi053 ceph-mon[36437]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:43.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:42 smithi062 ceph-mon[36507]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:44.856 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:44 smithi062 ceph-mon[36507]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:44.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:44 smithi046 ceph-mon[32038]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:44.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:44 smithi053 ceph-mon[36437]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:46.763 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:46 smithi062 ceph-mon[36507]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:46.763 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T07:52:46.763 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:46.763 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:46 smithi062 ceph-mon[36507]: Deploying daemon osd.8 on smithi062 2024-04-17T07:52:46.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:46 smithi046 ceph-mon[32038]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:46.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T07:52:46.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:46.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:46 smithi046 ceph-mon[32038]: Deploying daemon osd.8 on smithi062 2024-04-17T07:52:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:46 smithi053 ceph-mon[36437]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T07:52:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:46.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:46 smithi053 ceph-mon[36437]: Deploying daemon osd.8 on smithi062 2024-04-17T07:52:48.707 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:48 smithi062 ceph-mon[36507]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:48.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:48 smithi046 ceph-mon[32038]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:48.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:48 smithi053 ceph-mon[36437]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:50.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:50 smithi046 ceph-mon[32038]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:50.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:50 smithi062 ceph-mon[36507]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:50 smithi053 ceph-mon[36437]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:52 smithi053 ceph-mon[36437]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:52.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:52 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:52.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:52 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:52.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:52 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:52 smithi062 ceph-mon[36507]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:52 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:52 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:52 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:53.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:52 smithi046 ceph-mon[32038]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:53.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:52 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:53.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:52 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:53.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:52 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:54 smithi062 ceph-mon[36507]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:54 smithi053 ceph-mon[36437]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:55.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:54 smithi046 ceph-mon[32038]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.649+0000 7f774cff9700 1 -- 172.21.15.62:0/1758616132 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7750062e90 con 0x7f773c0a3c40 2024-04-17T07:52:55.650 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 8 on host 'smithi062' 2024-04-17T07:52:55.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f773c0a3c40 msgr2=0x7f773c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f773c0a3c40 0x7f773c0a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f774800b070 tx=0x7f77480060c0 comp rx=0 tx=0).stop 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750105e60 msgr2=0x7f7750199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750105e60 0x7f7750199290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f773800b560 tx=0x7f773800b920 comp rx=0 tx=0).stop 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 shutdown_connections 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f773c0a3c40 0x7f773c0a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750108790 0x7f77501a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7750103530 0x7f7750198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 --2- 172.21.15.62:0/1758616132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7750105e60 0x7f7750199290 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 >> 172.21.15.62:0/1758616132 conn(0x7f77500fcf40 msgr2=0x7f7750103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 shutdown_connections 2024-04-17T07:52:55.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:52:55.652+0000 7f7755ec6700 1 -- 172.21.15.62:0/1758616132 wait complete. 2024-04-17T07:52:56.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:56.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:56.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:52:56.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:52:56.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:56.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:55 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:56.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:56.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:52:56.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:52:56.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:56.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:55 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:52:56.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:52:56.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:52:56.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:52:56.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:52:56.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:55 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:52:56.356 DEBUG:teuthology.orchestra.run.smithi062:osd.8> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.8.service 2024-04-17T07:52:56.359 INFO:tasks.cephadm:Deploying osd.9 on smithi046 with /dev/vg_nvme/lv_1... 2024-04-17T07:52:56.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T07:52:56.840 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:52:57.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:56 smithi053 ceph-mon[36437]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:57.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:56 smithi053 ceph-mon[36437]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T07:52:57.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:56 smithi053 ceph-mon[36437]: from='osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:52:57.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:56 smithi053 ceph-mon[36437]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T07:52:57.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:56 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:57.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:56 smithi053 ceph-mon[36437]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:52:57.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:56 smithi062 ceph-mon[36507]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:57.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:56 smithi062 ceph-mon[36507]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T07:52:57.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:56 smithi062 ceph-mon[36507]: from='osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:52:57.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:56 smithi062 ceph-mon[36507]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T07:52:57.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:56 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:57.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:56 smithi062 ceph-mon[36507]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:52:57.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:56 smithi046 ceph-mon[32038]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:57.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:56 smithi046 ceph-mon[32038]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T07:52:57.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:56 smithi046 ceph-mon[32038]: from='osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:52:57.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:56 smithi046 ceph-mon[32038]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T07:52:57.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:56 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:57.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:56 smithi046 ceph-mon[32038]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:52:57.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 07:52:57 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T07:52:57.589+0000 7f2c2ee40700 -1 osd.8 0 waiting for initial osdmap 2024-04-17T07:52:57.804 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 07:52:57 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T07:52:57.603+0000 7f2c26c2b700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:52:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:57 smithi053 ceph-mon[36437]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:52:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:57 smithi053 ceph-mon[36437]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T07:52:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:57 smithi053 ceph-mon[36437]: from='osd.8 ' entity='osd.8' 2024-04-17T07:52:58.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:57 smithi062 ceph-mon[36507]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:52:58.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:57 smithi062 ceph-mon[36507]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T07:52:58.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:58.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:58.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:57 smithi062 ceph-mon[36507]: from='osd.8 ' entity='osd.8' 2024-04-17T07:52:58.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:57 smithi046 ceph-mon[32038]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:52:58.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:57 smithi046 ceph-mon[32038]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T07:52:58.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:58.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:58.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:57 smithi046 ceph-mon[32038]: from='osd.8 ' entity='osd.8' 2024-04-17T07:52:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:58 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:52:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:58 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:52:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:58 smithi062 ceph-mon[36507]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:58 smithi062 ceph-mon[36507]: osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] boot 2024-04-17T07:52:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:58 smithi062 ceph-mon[36507]: osdmap e59: 9 total, 9 up, 9 in 2024-04-17T07:52:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:58 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:52:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:58 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:52:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:58 smithi046 ceph-mon[32038]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:58 smithi046 ceph-mon[32038]: osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] boot 2024-04-17T07:52:59.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:58 smithi046 ceph-mon[32038]: osdmap e59: 9 total, 9 up, 9 in 2024-04-17T07:52:59.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:58 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:52:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:58 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:52:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:58 smithi053 ceph-mon[36437]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:58 smithi053 ceph-mon[36437]: osd.8 [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] boot 2024-04-17T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:58 smithi053 ceph-mon[36437]: osdmap e59: 9 total, 9 up, 9 in 2024-04-17T07:52:59.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:53:00.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T07:53:00.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:00.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:00.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:52:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T07:53:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T07:53:00.076 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-17T07:53:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-17T07:53:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0362458 s, 289 MB/s 2024-04-17T07:53:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.178 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:00.179 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:00.179 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:52:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:00.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:52:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:01.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-04-17T07:53:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:00 smithi053 ceph-mon[36437]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:00 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi062 2024-04-17T07:53:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:00 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi062 to 5693M 2024-04-17T07:53:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:00 smithi053 ceph-mon[36437]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T07:53:01.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:00 smithi046 ceph-mon[32038]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:01.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:00 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi062 2024-04-17T07:53:01.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:00 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi062 to 5693M 2024-04-17T07:53:01.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:00 smithi046 ceph-mon[32038]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T07:53:01.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:00 smithi062 ceph-mon[36507]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:01.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:00 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi062 2024-04-17T07:53:01.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:00 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi062 to 5693M 2024-04-17T07:53:01.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:00 smithi062 ceph-mon[36507]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T07:53:01.623 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:53:02.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:01 smithi046 ceph-mon[32038]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T07:53:02.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:01 smithi053 ceph-mon[36437]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T07:53:02.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:01 smithi062 ceph-mon[36507]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T07:53:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:02 smithi053 ceph-mon[36437]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:02 smithi053 ceph-mon[36437]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T07:53:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:02 smithi062 ceph-mon[36507]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:02 smithi062 ceph-mon[36507]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T07:53:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:02 smithi046 ceph-mon[32038]: pgmap v222: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:02 smithi046 ceph-mon[32038]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T07:53:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 -- 172.21.15.46:0/2754262314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304102ae0 msgr2=0x7f5304102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 --2- 172.21.15.46:0/2754262314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304102ae0 0x7f5304102f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009990 tx=0x7f52f4009ca0 comp rx=0 tx=0).stop 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 -- 172.21.15.46:0/2754262314 shutdown_connections 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 --2- 172.21.15.46:0/2754262314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304104680 0x7f530410af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 --2- 172.21.15.46:0/2754262314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304102ae0 0x7f5304102f00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 --2- 172.21.15.46:0/2754262314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304103ce0 0x7f5304104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 -- 172.21.15.46:0/2754262314 >> 172.21.15.46:0/2754262314 conn(0x7f53040fe0a0 msgr2=0x7f53041004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 -- 172.21.15.46:0/2754262314 shutdown_connections 2024-04-17T07:53:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.952+0000 7f530ac00700 1 -- 172.21.15.46:0/2754262314 wait complete. 2024-04-17T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.953+0000 7f530ac00700 1 Processor -- start 2024-04-17T07:53:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f530ac00700 1 -- start start 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f530ac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304102ae0 0x7f530419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f530ac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304103ce0 0x7f530419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f530ac00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304104680 0x7f53041a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f530ac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f530410e710 con 0x7f5304102ae0 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f530ac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f530410e590 con 0x7f5304103ce0 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f530ac00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f530410e890 con 0x7f5304104680 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f5303e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304102ae0 0x7f530419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f52fb400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304103ce0 0x7f530419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f5303e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304102ae0 0x7f530419af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50084/0 (socket says 172.21.15.46:50084) 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f5303e00700 1 -- 172.21.15.46:0/1435371924 learned_addr learned my addr 172.21.15.46:0/1435371924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.954+0000 7f5308c00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304104680 0x7f53041a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f52fb400700 1 -- 172.21.15.46:0/1435371924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304104680 msgr2=0x7f53041a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f52fb400700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304104680 0x7f53041a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f52fb400700 1 -- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304102ae0 msgr2=0x7f530419af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f52fb400700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304102ae0 0x7f530419af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f52fb400700 1 -- 172.21.15.46:0/1435371924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53041a2bd0 con 0x7f5304103ce0 2024-04-17T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f5303e00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304102ae0 0x7f530419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:53:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f5308c00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304104680 0x7f53041a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.955+0000 7f52fb400700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304103ce0 0x7f530419b4b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00e9f0 tx=0x7f52ec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.956+0000 7f5301600700 1 -- 172.21.15.46:0/1435371924 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec004220 con 0x7f5304103ce0 2024-04-17T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.956+0000 7f5301600700 1 -- 172.21.15.46:0/1435371924 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ec004d10 con 0x7f5304103ce0 2024-04-17T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.956+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53041a2ec0 con 0x7f5304103ce0 2024-04-17T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.956+0000 7f5301600700 1 -- 172.21.15.46:0/1435371924 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec005020 con 0x7f5304103ce0 2024-04-17T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.956+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a3370 con 0x7f5304103ce0 2024-04-17T07:53:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.957+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5304069770 con 0x7f5304103ce0 2024-04-17T07:53:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.959+0000 7f5301600700 1 -- 172.21.15.46:0/1435371924 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f52ec004750 con 0x7f5304103ce0 2024-04-17T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.960+0000 7f5301600700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e80a3cc0 0x7f52e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.960+0000 7f5301600700 1 -- 172.21.15.46:0/1435371924 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f52ec0cf8e0 con 0x7f5304103ce0 2024-04-17T07:53:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.961+0000 7f5303e00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e80a3cc0 0x7f52e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.961+0000 7f5303e00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e80a3cc0 0x7f52e80a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52f400b2e0 tx=0x7f52f401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:53:03.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:03.965+0000 7f5301600700 1 -- 172.21.15.46:0/1435371924 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5304069770 con 0x7f5304103ce0 2024-04-17T07:53:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:04.208+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f53040763a0 con 0x7f52e80a3cc0 2024-04-17T07:53:05.206 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:04 smithi046 ceph-mon[32038]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-17T07:53:05.207 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:04 smithi046 ceph-mon[32038]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:53:05.207 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:53:05.207 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:53:05.207 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:04 smithi053 ceph-mon[36437]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-17T07:53:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:04 smithi053 ceph-mon[36437]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:53:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:53:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:53:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:04 smithi062 ceph-mon[36507]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-17T07:53:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:04 smithi062 ceph-mon[36507]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:53:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:53:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:53:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:07.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:06 smithi046 ceph-mon[32038]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:53:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:06 smithi053 ceph-mon[36437]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:53:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:06 smithi062 ceph-mon[36507]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:53:09.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:08 smithi053 ceph-mon[36437]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-17T07:53:09.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:08 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2301186964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f44d01-ea3b-4b7a-a763-8d39e757395b"}]: dispatch 2024-04-17T07:53:09.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:08 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2301186964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f44d01-ea3b-4b7a-a763-8d39e757395b"}]': finished 2024-04-17T07:53:09.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:08 smithi053 ceph-mon[36437]: osdmap e64: 10 total, 9 up, 10 in 2024-04-17T07:53:09.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:08 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:08 smithi062 ceph-mon[36507]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-17T07:53:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:08 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2301186964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f44d01-ea3b-4b7a-a763-8d39e757395b"}]: dispatch 2024-04-17T07:53:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:08 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2301186964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f44d01-ea3b-4b7a-a763-8d39e757395b"}]': finished 2024-04-17T07:53:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:08 smithi062 ceph-mon[36507]: osdmap e64: 10 total, 9 up, 10 in 2024-04-17T07:53:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:08 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:09.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:08 smithi046 ceph-mon[32038]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-17T07:53:09.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:08 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2301186964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f44d01-ea3b-4b7a-a763-8d39e757395b"}]: dispatch 2024-04-17T07:53:09.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:08 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2301186964' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f44d01-ea3b-4b7a-a763-8d39e757395b"}]': finished 2024-04-17T07:53:09.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:08 smithi046 ceph-mon[32038]: osdmap e64: 10 total, 9 up, 10 in 2024-04-17T07:53:09.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:08 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:10.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:09 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2192172118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:53:10.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:09 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2192172118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:53:10.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:09 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2192172118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:53:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:10 smithi062 ceph-mon[36507]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:53:11.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:10 smithi046 ceph-mon[32038]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:53:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:10 smithi053 ceph-mon[36437]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:53:13.273 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:12 smithi053 ceph-mon[36437]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-17T07:53:13.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:12 smithi062 ceph-mon[36507]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-17T07:53:13.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:12 smithi046 ceph-mon[32038]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-17T07:53:15.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:14 smithi062 ceph-mon[36507]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:15.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:14 smithi046 ceph-mon[32038]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:15.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:14 smithi053 ceph-mon[36437]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:17.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:16 smithi046 ceph-mon[32038]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:17.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:16 smithi062 ceph-mon[36507]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:17.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:16 smithi053 ceph-mon[36437]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:18 smithi062 ceph-mon[36507]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:18 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T07:53:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:18 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:18 smithi062 ceph-mon[36507]: Deploying daemon osd.9 on smithi046 2024-04-17T07:53:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:18 smithi046 ceph-mon[32038]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:18 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T07:53:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:18 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:18 smithi046 ceph-mon[32038]: Deploying daemon osd.9 on smithi046 2024-04-17T07:53:19.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:18 smithi053 ceph-mon[36437]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:19.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:18 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T07:53:19.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:18 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:19.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:18 smithi053 ceph-mon[36437]: Deploying daemon osd.9 on smithi046 2024-04-17T07:53:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:20 smithi062 ceph-mon[36507]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:21.314 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:20 smithi046 ceph-mon[32038]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:21.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:20 smithi053 ceph-mon[36437]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:23.275 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:23 smithi046 ceph-mon[32038]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:23 smithi062 ceph-mon[36507]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:23 smithi053 ceph-mon[36437]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:25.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:25 smithi046 ceph-mon[32038]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:25.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:25 smithi062 ceph-mon[36507]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:25 smithi053 ceph-mon[36437]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:27.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:27 smithi046 ceph-mon[32038]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:27.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:27 smithi062 ceph-mon[36507]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:27 smithi053 ceph-mon[36437]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:28.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:28.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:28.344 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:28.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:28.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:28.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:29.334 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:29 smithi046 ceph-mon[32038]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:29 smithi053 ceph-mon[36437]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:29.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:29 smithi062 ceph-mon[36507]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:31.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:31 smithi046 ceph-mon[32038]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:31.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:31.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:31.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:31 smithi046 ceph-mon[32038]: from='osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:53:31.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:31 smithi046 ceph-mon[32038]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:31 smithi053 ceph-mon[36437]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:31 smithi053 ceph-mon[36437]: from='osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:53:31.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:31 smithi053 ceph-mon[36437]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:53:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:31 smithi062 ceph-mon[36507]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:31 smithi062 ceph-mon[36507]: from='osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:53:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:31 smithi062 ceph-mon[36507]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:53:32.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.163+0000 7f5301600700 1 -- 172.21.15.46:0/1435371924 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53040763a0 con 0x7f52e80a3cc0 2024-04-17T07:53:32.164 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 9 on host 'smithi046' 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e80a3cc0 msgr2=0x7f52e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e80a3cc0 0x7f52e80a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52f400b2e0 tx=0x7f52f401a040 comp rx=0 tx=0).stop 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304103ce0 msgr2=0x7f530419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304103ce0 0x7f530419b4b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00e9f0 tx=0x7f52ec00ed00 comp rx=0 tx=0).stop 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 shutdown_connections 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e80a3cc0 0x7f52e80a6180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5304104680 0x7f53041a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5304102ae0 0x7f530419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 --2- 172.21.15.46:0/1435371924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304103ce0 0x7f530419b4b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 >> 172.21.15.46:0/1435371924 conn(0x7f53040fe0a0 msgr2=0x7f5304074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:53:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 shutdown_connections 2024-04-17T07:53:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:53:32.167+0000 7f530ac00700 1 -- 172.21.15.46:0/1435371924 wait complete. 2024-04-17T07:53:32.989 DEBUG:teuthology.orchestra.run.smithi046:osd.9> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.9.service 2024-04-17T07:53:32.997 INFO:tasks.cephadm:Deploying osd.10 on smithi053 with /dev/vg_nvme/lv_1... 2024-04-17T07:53:32.997 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T07:53:33.260 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:53:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T07:53:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: from='osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:53:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T07:53:33.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:53:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:33.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:33.361 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T07:53:33.109+0000 7f951c400700 -1 osd.9 0 waiting for initial osdmap 2024-04-17T07:53:33.361 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 07:53:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T07:53:33.148+0000 7f9517200700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:53:33.382 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:33.382 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T07:53:33.382 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: from='osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:53:33.382 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T07:53:33.382 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:33.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:53:33.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:33.383 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:33 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:33.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T07:53:33.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: from='osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:53:33.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T07:53:33.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:33.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-17T07:53:33.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:33.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:33 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:34.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:34 smithi046 ceph-mon[32038]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:53:34.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:34 smithi046 ceph-mon[32038]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T07:53:34.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:34.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:34 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:34.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:34 smithi053 ceph-mon[36437]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:53:34.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:34 smithi053 ceph-mon[36437]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T07:53:34.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:34.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:34 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:34.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:34 smithi062 ceph-mon[36507]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-17T07:53:34.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:34 smithi062 ceph-mon[36507]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T07:53:34.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:34.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:34 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:35.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:35 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:53:35.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:35 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:53:35.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:35 smithi046 ceph-mon[32038]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:35.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:35 smithi046 ceph-mon[32038]: osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] boot 2024-04-17T07:53:35.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:35 smithi046 ceph-mon[32038]: osdmap e67: 10 total, 10 up, 10 in 2024-04-17T07:53:35.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:35 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:35.454 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:35 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:53:35.455 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:35 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:53:35.455 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:35 smithi062 ceph-mon[36507]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:35.455 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:35 smithi062 ceph-mon[36507]: osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] boot 2024-04-17T07:53:35.455 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:35 smithi062 ceph-mon[36507]: osdmap e67: 10 total, 10 up, 10 in 2024-04-17T07:53:35.455 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:35 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:35.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:35 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:53:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:35 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:53:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:35 smithi053 ceph-mon[36437]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:53:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:35 smithi053 ceph-mon[36437]: osd.9 [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] boot 2024-04-17T07:53:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:35 smithi053 ceph-mon[36437]: osdmap e67: 10 total, 10 up, 10 in 2024-04-17T07:53:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:35 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:53:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T07:53:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T07:53:35.535 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-04-17T07:53:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-04-17T07:53:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00835964 s, 1.3 GB/s 2024-04-17T07:53:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-04-17T07:53:36.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-04-17T07:53:36.530 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:53:37.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:37 smithi046 ceph-mon[32038]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:37.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:37 smithi046 ceph-mon[32038]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T07:53:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:37 smithi053 ceph-mon[36437]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:37 smithi053 ceph-mon[36437]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T07:53:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:37 smithi062 ceph-mon[36507]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:37 smithi062 ceph-mon[36507]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T07:53:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 -- 172.21.15.53:0/4197773957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d41019b0 msgr2=0x7f89d4101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 --2- 172.21.15.53:0/4197773957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d41019b0 0x7f89d4101dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89c80099c0 tx=0x7f89c8009cd0 comp rx=0 tx=0).stop 2024-04-17T07:53:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 -- 172.21.15.53:0/4197773957 shutdown_connections 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 --2- 172.21.15.53:0/4197773957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d4103550 0x7f89d4109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 --2- 172.21.15.53:0/4197773957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d41019b0 0x7f89d4101dd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 --2- 172.21.15.53:0/4197773957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d4102bb0 0x7f89d4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 -- 172.21.15.53:0/4197773957 >> 172.21.15.53:0/4197773957 conn(0x7f89d40fcf50 msgr2=0x7f89d40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 -- 172.21.15.53:0/4197773957 shutdown_connections 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.205+0000 7f89d88df700 1 -- 172.21.15.53:0/4197773957 wait complete. 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 Processor -- start 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 -- start start 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d41019b0 0x7f89d41058f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d4102bb0 0x7f89d4103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d4103550 0x7f89d41044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410d610 con 0x7f89d4103550 2024-04-17T07:53:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410d490 con 0x7f89d4102bb0 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d88df700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410d790 con 0x7f89d41019b0 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d41019b0 0x7f89d41058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d41019b0 0x7f89d41058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.53:36510/0 (socket says 172.21.15.53:36510) 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d259c700 1 -- 172.21.15.53:0/2654422891 learned_addr learned my addr 172.21.15.53:0/2654422891 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.206+0000 7f89d1d9b700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d4102bb0 0x7f89d4103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d1d9b700 1 -- 172.21.15.53:0/2654422891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d41019b0 msgr2=0x7f89d41058f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d1d9b700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d41019b0 0x7f89d41058f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d1d9b700 1 -- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d4103550 msgr2=0x7f89d41044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d2d9d700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d4103550 0x7f89d41044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d1d9b700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d4103550 0x7f89d41044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d1d9b700 1 -- 172.21.15.53:0/2654422891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d4104da0 con 0x7f89d4102bb0 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d259c700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d41019b0 0x7f89d41058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:53:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d1d9b700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d4102bb0 0x7f89d4103f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c970 tx=0x7f89c400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:53:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89c37fe700 1 -- 172.21.15.53:0/2654422891 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c40078f0 con 0x7f89d4102bb0 2024-04-17T07:53:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89c37fe700 1 -- 172.21.15.53:0/2654422891 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c400f450 con 0x7f89d4102bb0 2024-04-17T07:53:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41a2ee0 con 0x7f89d4102bb0 2024-04-17T07:53:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89c37fe700 1 -- 172.21.15.53:0/2654422891 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4018660 con 0x7f89d4102bb0 2024-04-17T07:53:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.207+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d41a3360 con 0x7f89d4102bb0 2024-04-17T07:53:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.208+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d4069740 con 0x7f89d4102bb0 2024-04-17T07:53:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.209+0000 7f89c37fe700 1 -- 172.21.15.53:0/2654422891 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f89c400f600 con 0x7f89d4102bb0 2024-04-17T07:53:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.210+0000 7f89c37fe700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89bc0a3cc0 0x7f89bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:53:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.210+0000 7f89c37fe700 1 -- 172.21.15.53:0/2654422891 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f89c40cfae0 con 0x7f89d4102bb0 2024-04-17T07:53:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.211+0000 7f89d259c700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89bc0a3cc0 0x7f89bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:53:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.211+0000 7f89d259c700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89bc0a3cc0 0x7f89bc0a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009fd0 tx=0x7f89c8003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:53:38.214 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.215+0000 7f89c37fe700 1 -- 172.21.15.53:0/2654422891 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f89c4014030 con 0x7f89d4102bb0 2024-04-17T07:53:38.433 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:38.433+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f89d40ff140 con 0x7f89bc0a3cc0 2024-04-17T07:53:39.425 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:39.425 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi046 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:53:39.426 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:53:39.427 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.427 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:39.427 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.427 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:39.427 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi046 2024-04-17T07:53:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:39.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi046 2024-04-17T07:53:39.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:39.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:53:39.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:40.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:40 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi046 to 4273M 2024-04-17T07:53:40.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:40 smithi062 ceph-mon[36507]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:53:40.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:40 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi046 to 4273M 2024-04-17T07:53:40.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:40 smithi046 ceph-mon[32038]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:53:40.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:40 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi046 to 4273M 2024-04-17T07:53:40.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:40 smithi053 ceph-mon[36437]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:53:41.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:41 smithi062 ceph-mon[36507]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:41.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:41 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/134992314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]: dispatch 2024-04-17T07:53:41.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:41 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]: dispatch 2024-04-17T07:53:41.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:41 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]': finished 2024-04-17T07:53:41.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:41 smithi062 ceph-mon[36507]: osdmap e69: 11 total, 10 up, 11 in 2024-04-17T07:53:41.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:41 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:53:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:41 smithi046 ceph-mon[32038]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:41 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/134992314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]: dispatch 2024-04-17T07:53:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:41 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]: dispatch 2024-04-17T07:53:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:41 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]': finished 2024-04-17T07:53:41.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:41 smithi046 ceph-mon[32038]: osdmap e69: 11 total, 10 up, 11 in 2024-04-17T07:53:41.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:41 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:53:41.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:41 smithi053 ceph-mon[36437]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:41.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:41 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/134992314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]: dispatch 2024-04-17T07:53:41.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:41 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]: dispatch 2024-04-17T07:53:41.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:41 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93b1b906-94d1-4af6-8ab1-b69157e8185c"}]': finished 2024-04-17T07:53:41.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:41 smithi053 ceph-mon[36437]: osdmap e69: 11 total, 10 up, 11 in 2024-04-17T07:53:41.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:41 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:53:42.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:42 smithi062 ceph-mon[36507]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:42.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:42 smithi062 ceph-mon[36507]: from='client.? 172.21.15.53:0/10749795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:53:42.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:42 smithi046 ceph-mon[32038]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:42.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:42 smithi046 ceph-mon[32038]: from='client.? 172.21.15.53:0/10749795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:53:42.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:42 smithi053 ceph-mon[36437]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:42.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:42 smithi053 ceph-mon[36437]: from='client.? 172.21.15.53:0/10749795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:53:44.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:44 smithi053 ceph-mon[36437]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:44.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:44 smithi062 ceph-mon[36507]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:44.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:44 smithi046 ceph-mon[32038]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:46.777 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:46 smithi062 ceph-mon[36507]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:46.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:46 smithi046 ceph-mon[32038]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:46 smithi053 ceph-mon[36437]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:48.631 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:48 smithi053 ceph-mon[36437]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:48.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:48 smithi062 ceph-mon[36507]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:48.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:48 smithi046 ceph-mon[32038]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:49.691 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T07:53:49.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:49.692 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:49 smithi053 ceph-mon[36437]: Deploying daemon osd.10 on smithi053 2024-04-17T07:53:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T07:53:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:49 smithi062 ceph-mon[36507]: Deploying daemon osd.10 on smithi053 2024-04-17T07:53:49.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T07:53:49.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:53:49.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:49 smithi046 ceph-mon[32038]: Deploying daemon osd.10 on smithi053 2024-04-17T07:53:50.668 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:50 smithi053 ceph-mon[36437]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:50.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:50 smithi062 ceph-mon[36507]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:50.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:50 smithi046 ceph-mon[32038]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:52.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:52 smithi062 ceph-mon[36507]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:52.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:52 smithi046 ceph-mon[32038]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:52 smithi053 ceph-mon[36437]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:54 smithi046 ceph-mon[32038]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:54.901 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:54 smithi053 ceph-mon[36437]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:54.902 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:54.902 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:54.902 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:54 smithi062 ceph-mon[36507]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:53:56.824 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:56 smithi053 ceph-mon[36437]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:56.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:56 smithi046 ceph-mon[32038]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:57.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:56 smithi062 ceph-mon[36507]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:57.872 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 10 on host 'smithi053' 2024-04-17T07:53:57.872 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.871+0000 7f89c37fe700 1 -- 172.21.15.53:0/2654422891 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f89d40ff140 con 0x7f89bc0a3cc0 2024-04-17T07:53:57.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89bc0a3cc0 msgr2=0x7f89bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:57.874 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89bc0a3cc0 0x7f89bc0a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009fd0 tx=0x7f89c8003960 comp rx=0 tx=0).stop 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d4102bb0 msgr2=0x7f89d4103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d4102bb0 0x7f89d4103f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c970 tx=0x7f89c400cc80 comp rx=0 tx=0).stop 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 shutdown_connections 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89bc0a3cc0 0x7f89bc0a6180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d4103550 0x7f89d41044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89d41019b0 0x7f89d41058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 --2- 172.21.15.53:0/2654422891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d4102bb0 0x7f89d4103f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:53:57.875 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 >> 172.21.15.53:0/2654422891 conn(0x7f89d40fcf50 msgr2=0x7f89d40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:53:57.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 shutdown_connections 2024-04-17T07:53:57.876 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-17T07:53:57.874+0000 7f89d88df700 1 -- 172.21.15.53:0/2654422891 wait complete. 2024-04-17T07:53:58.564 DEBUG:teuthology.orchestra.run.smithi053:osd.10> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.10.service 2024-04-17T07:53:58.567 INFO:tasks.cephadm:Deploying osd.11 on smithi062 with /dev/vg_nvme/lv_1... 2024-04-17T07:53:58.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T07:53:58.830 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:53:58.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:58 smithi062 ceph-mon[36507]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:58.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:58.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:58.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:58 smithi062 ceph-mon[36507]: from='osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:53:58.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:58 smithi062 ceph-mon[36507]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:53:58.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:58.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:58 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:58 smithi046 ceph-mon[32038]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:58 smithi046 ceph-mon[32038]: from='osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:53:59.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:58 smithi046 ceph-mon[32038]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:53:59.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:58 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:58 smithi053 ceph-mon[36437]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:53:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:58 smithi053 ceph-mon[36437]: from='osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:53:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:58 smithi053 ceph-mon[36437]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:53:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:58 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:53:59.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:59 smithi053 ceph-mon[36437]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T07:53:59.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:59 smithi053 ceph-mon[36437]: from='osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:53:59.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:59 smithi053 ceph-mon[36437]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T07:53:59.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:59 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:53:59.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:53:59 smithi053 ceph-mon[36437]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:53:59.972 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 07:53:59 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T07:53:59.834+0000 7f294be0f700 -1 osd.10 0 waiting for initial osdmap 2024-04-17T07:53:59.972 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 07:53:59 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T07:53:59.847+0000 7f2947c02700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:54:00.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:59 smithi046 ceph-mon[32038]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T07:54:00.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:59 smithi046 ceph-mon[32038]: from='osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:54:00.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:59 smithi046 ceph-mon[32038]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T07:54:00.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:59 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:00.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:53:59 smithi046 ceph-mon[32038]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:54:00.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:59 smithi062 ceph-mon[36507]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T07:54:00.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:59 smithi062 ceph-mon[36507]: from='osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:54:00.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:59 smithi062 ceph-mon[36507]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T07:54:00.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:59 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:00.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:53:59 smithi062 ceph-mon[36507]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-04-17T07:54:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:00 smithi046 ceph-mon[32038]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:54:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:00 smithi046 ceph-mon[32038]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:54:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:00 smithi046 ceph-mon[32038]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T07:54:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:01.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:00 smithi062 ceph-mon[36507]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:54:01.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:00 smithi062 ceph-mon[36507]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:54:01.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:00 smithi062 ceph-mon[36507]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T07:54:01.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:01.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T07:54:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T07:54:01.119 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-17T07:54:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-17T07:54:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137999 s, 760 MB/s 2024-04-17T07:54:01.119 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 2024-04-17T07:54:01.119 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-17T07:54:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:00 smithi053 ceph-mon[36437]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:54:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:00 smithi053 ceph-mon[36437]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-04-17T07:54:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:00 smithi053 ceph-mon[36437]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T07:54:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:01.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-04-17T07:54:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:54:02.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:01 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:54:02.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:01 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:54:02.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:01 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:02.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:01 smithi062 ceph-mon[36507]: osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] boot 2024-04-17T07:54:02.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:01 smithi062 ceph-mon[36507]: osdmap e72: 11 total, 11 up, 11 in 2024-04-17T07:54:02.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:01 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:02.086 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:01 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:54:02.086 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:01 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:54:02.086 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:01 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:02.086 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:01 smithi053 ceph-mon[36437]: osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] boot 2024-04-17T07:54:02.086 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:01 smithi053 ceph-mon[36437]: osdmap e72: 11 total, 11 up, 11 in 2024-04-17T07:54:02.086 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:01 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:02.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:01 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:54:02.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:01 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:54:02.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:01 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:02.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:01 smithi046 ceph-mon[32038]: osd.10 [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] boot 2024-04-17T07:54:02.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:01 smithi046 ceph-mon[32038]: osdmap e72: 11 total, 11 up, 11 in 2024-04-17T07:54:02.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:01 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:54:03.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:54:03.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T07:54:03.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi053 2024-04-17T07:54:03.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi053 to 4271M 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:03.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:03.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:02 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:54:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T07:54:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi053 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi053 to 4271M 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:03.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:03.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:03.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:02 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:54:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T07:54:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi053 2024-04-17T07:54:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi053 to 4271M 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:03.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:02 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:03.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.679+0000 7f2f79f3b700 1 -- 172.21.15.62:0/278537839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74104690 msgr2=0x7f2f7410af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.679+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/278537839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74104690 0x7f2f7410af40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00b0a0 tx=0x7f2f6c00b3b0 comp rx=0 tx=0).stop 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 -- 172.21.15.62:0/278537839 shutdown_connections 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/278537839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74104690 0x7f2f7410af40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/278537839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f74102af0 0x7f2f74102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/278537839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f74103cf0 0x7f2f74104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 -- 172.21.15.62:0/278537839 >> 172.21.15.62:0/278537839 conn(0x7f2f740fe090 msgr2=0x7f2f741004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 -- 172.21.15.62:0/278537839 shutdown_connections 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 -- 172.21.15.62:0/278537839 wait complete. 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 Processor -- start 2024-04-17T07:54:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.680+0000 7f2f79f3b700 1 -- start start 2024-04-17T07:54:03.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f79f3b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f74102af0 0x7f2f74198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:03.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f79f3b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74103cf0 0x7f2f74199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:03.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f79f3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f74104690 0x7f2f741a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f79f3b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7410e7d0 con 0x7f2f74104690 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f79f3b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7410e650 con 0x7f2f74102af0 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f79f3b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7410e950 con 0x7f2f74103cf0 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74103cf0 0x7f2f74199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74103cf0 0x7f2f74199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50420/0 (socket says 172.21.15.62:50420) 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 -- 172.21.15.62:0/3148250404 learned_addr learned my addr 172.21.15.62:0/3148250404 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f737fe700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f74102af0 0x7f2f74198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 -- 172.21.15.62:0/3148250404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f74102af0 msgr2=0x7f2f74198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f74102af0 0x7f2f74198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:03.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 -- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f74104690 msgr2=0x7f2f741a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f74104690 0x7f2f741a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 -- 172.21.15.62:0/3148250404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f741a0a50 con 0x7f2f74103cf0 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f737fe700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f74102af0 0x7f2f74198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f72ffd700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74103cf0 0x7f2f74199330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800b560 tx=0x7f2f6800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.681+0000 7f2f70ff9700 1 -- 172.21.15.62:0/3148250404 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68015020 con 0x7f2f74103cf0 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.682+0000 7f2f70ff9700 1 -- 172.21.15.62:0/3148250404 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f68011d10 con 0x7f2f74103cf0 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.682+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f741a0d40 con 0x7f2f74103cf0 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.682+0000 7f2f70ff9700 1 -- 172.21.15.62:0/3148250404 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800f2e0 con 0x7f2f74103cf0 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.682+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f741a12f0 con 0x7f2f74103cf0 2024-04-17T07:54:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.682+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7402df90 con 0x7f2f74103cf0 2024-04-17T07:54:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.684+0000 7f2f70ff9700 1 -- 172.21.15.62:0/3148250404 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2f68018020 con 0x7f2f74103cf0 2024-04-17T07:54:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.684+0000 7f2f70ff9700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2f5c0a3cc0 0x7f2f5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.684+0000 7f2f70ff9700 1 -- 172.21.15.62:0/3148250404 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f2f680cf570 con 0x7f2f74103cf0 2024-04-17T07:54:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.685+0000 7f2f737fe700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2f5c0a3cc0 0x7f2f5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.686+0000 7f2f737fe700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2f5c0a3cc0 0x7f2f5c0a6180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f74103b50 tx=0x7f2f64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:03.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.689+0000 7f2f70ff9700 1 -- 172.21.15.62:0/3148250404 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2f6801f020 con 0x7f2f74103cf0 2024-04-17T07:54:03.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:03.910+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2f74062e90 con 0x7f2f5c0a3cc0 2024-04-17T07:54:04.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:03 smithi053 ceph-mon[36437]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T07:54:04.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:03 smithi062 ceph-mon[36507]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T07:54:04.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:03 smithi046 ceph-mon[32038]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T07:54:05.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:04 smithi046 ceph-mon[32038]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:05.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:04 smithi046 ceph-mon[32038]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T07:54:05.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:04 smithi046 ceph-mon[32038]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:54:05.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:54:05.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:54:05.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:05.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:04 smithi053 ceph-mon[36437]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:05.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:04 smithi053 ceph-mon[36437]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T07:54:05.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:04 smithi053 ceph-mon[36437]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:54:05.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:54:05.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:54:05.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:04 smithi062 ceph-mon[36507]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:04 smithi062 ceph-mon[36507]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T07:54:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:04 smithi062 ceph-mon[36507]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:54:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:54:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:54:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:06.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:05 smithi053 ceph-mon[36437]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T07:54:06.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:05 smithi062 ceph-mon[36507]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T07:54:06.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:05 smithi046 ceph-mon[32038]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T07:54:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:06 smithi053 ceph-mon[36437]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:06 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/569290954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]: dispatch 2024-04-17T07:54:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:06 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]: dispatch 2024-04-17T07:54:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:06 smithi053 ceph-mon[36437]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]': finished 2024-04-17T07:54:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:06 smithi053 ceph-mon[36437]: osdmap e77: 12 total, 11 up, 12 in 2024-04-17T07:54:07.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:06 smithi062 ceph-mon[36507]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:06 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/569290954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]: dispatch 2024-04-17T07:54:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:06 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]: dispatch 2024-04-17T07:54:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:06 smithi062 ceph-mon[36507]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]': finished 2024-04-17T07:54:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:06 smithi062 ceph-mon[36507]: osdmap e77: 12 total, 11 up, 12 in 2024-04-17T07:54:07.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:07.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:06 smithi046 ceph-mon[32038]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:06 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/569290954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]: dispatch 2024-04-17T07:54:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:06 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]: dispatch 2024-04-17T07:54:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:06 smithi046 ceph-mon[32038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaf98df8-8674-48c4-87d5-531ef5a44098"}]': finished 2024-04-17T07:54:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:06 smithi046 ceph-mon[32038]: osdmap e77: 12 total, 11 up, 12 in 2024-04-17T07:54:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:07 smithi053 ceph-mon[36437]: from='client.? 172.21.15.62:0/3717380934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:54:08.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:07 smithi062 ceph-mon[36507]: from='client.? 172.21.15.62:0/3717380934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:54:08.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:07 smithi046 ceph-mon[32038]: from='client.? 172.21.15.62:0/3717380934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:54:09.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:08 smithi053 ceph-mon[36437]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:09.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:08 smithi062 ceph-mon[36507]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:09.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:08 smithi046 ceph-mon[32038]: pgmap v270: 1 pgs: 1 remapped+peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:10 smithi062 ceph-mon[36507]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:54:11.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:10 smithi053 ceph-mon[36437]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:54:11.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:10 smithi046 ceph-mon[32038]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:54:13.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:12 smithi062 ceph-mon[36507]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-17T07:54:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:12 smithi053 ceph-mon[36437]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-17T07:54:13.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:12 smithi046 ceph-mon[32038]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-17T07:54:15.192 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:14 smithi062 ceph-mon[36507]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-17T07:54:15.192 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:14 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T07:54:15.192 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:14 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:14 smithi053 ceph-mon[36437]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-17T07:54:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:14 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T07:54:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:14 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:14 smithi046 ceph-mon[32038]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-17T07:54:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:14 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T07:54:15.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:14 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:16.175 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:15 smithi062 ceph-mon[36507]: Deploying daemon osd.11 on smithi062 2024-04-17T07:54:16.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:15 smithi053 ceph-mon[36437]: Deploying daemon osd.11 on smithi062 2024-04-17T07:54:16.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:15 smithi046 ceph-mon[32038]: Deploying daemon osd.11 on smithi062 2024-04-17T07:54:17.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:16 smithi062 ceph-mon[36507]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:54:17.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:16 smithi053 ceph-mon[36437]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:54:17.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:16 smithi046 ceph-mon[32038]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:54:19.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:18 smithi062 ceph-mon[36507]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-17T07:54:19.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:18 smithi053 ceph-mon[36437]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-17T07:54:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:18 smithi046 ceph-mon[32038]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-17T07:54:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:21 smithi062 ceph-mon[36507]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-17T07:54:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:21.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:21.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:21 smithi046 ceph-mon[32038]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-17T07:54:21.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:21.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:21.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:21.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:21 smithi053 ceph-mon[36437]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-17T07:54:21.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:21.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:21.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:23 smithi062 ceph-mon[36507]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:23.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:23 smithi046 ceph-mon[32038]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:23 smithi053 ceph-mon[36437]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:24.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.049+0000 7f2f70ff9700 1 -- 172.21.15.62:0/3148250404 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2f74062e90 con 0x7f2f5c0a3cc0 2024-04-17T07:54:24.049 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 11 on host 'smithi062' 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2f5c0a3cc0 msgr2=0x7f2f5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2f5c0a3cc0 0x7f2f5c0a6180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f74103b50 tx=0x7f2f64008040 comp rx=0 tx=0).stop 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74103cf0 msgr2=0x7f2f74199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74103cf0 0x7f2f74199330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800b560 tx=0x7f2f6800b920 comp rx=0 tx=0).stop 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 shutdown_connections 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2f5c0a3cc0 0x7f2f5c0a6180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f74104690 0x7f2f741a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:24.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f74102af0 0x7f2f74198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:24.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 --2- 172.21.15.62:0/3148250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f74103cf0 0x7f2f74199330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:24.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.051+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 >> 172.21.15.62:0/3148250404 conn(0x7f2f740fe090 msgr2=0x7f2f74106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:24.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.052+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 shutdown_connections 2024-04-17T07:54:24.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:24.052+0000 7f2f79f3b700 1 -- 172.21.15.62:0/3148250404 wait complete. 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:24.736 DEBUG:teuthology.orchestra.run.smithi062:osd.11> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.11.service 2024-04-17T07:54:24.738 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-17T07:54:24.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd stat -f json 2024-04-17T07:54:25.221 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:54:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-mon[36507]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-mon[36507]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T07:54:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-mon[36507]: from='osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:54:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-mon[36507]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T07:54:25.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:25.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-mon[36507]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:54:25.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:25 smithi046 ceph-mon[32038]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:25.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:25 smithi046 ceph-mon[32038]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T07:54:25.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:25 smithi046 ceph-mon[32038]: from='osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:54:25.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:25 smithi046 ceph-mon[32038]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T07:54:25.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:25.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:25 smithi046 ceph-mon[32038]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:54:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:25 smithi053 ceph-mon[36437]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:25 smithi053 ceph-mon[36437]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T07:54:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:25 smithi053 ceph-mon[36437]: from='osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:54:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:25 smithi053 ceph-mon[36437]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T07:54:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:25 smithi053 ceph-mon[36437]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-17T07:54:25.957 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T07:54:25.810+0000 7f9f006b7700 -1 osd.11 0 waiting for initial osdmap 2024-04-17T07:54:25.957 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 07:54:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T07:54:25.827+0000 7f9ef7ca1700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:54:26.270 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:26 smithi062 ceph-mon[36507]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:54:26.270 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:26 smithi062 ceph-mon[36507]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T07:54:26.270 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:26.271 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:26.271 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:26 smithi062 ceph-mon[36507]: from='osd.11 ' entity='osd.11' 2024-04-17T07:54:26.398 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:26 smithi046 ceph-mon[32038]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:54:26.398 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:26 smithi046 ceph-mon[32038]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T07:54:26.399 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:26.399 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:26.399 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:26 smithi046 ceph-mon[32038]: from='osd.11 ' entity='osd.11' 2024-04-17T07:54:26.475 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:26 smithi053 ceph-mon[36437]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-17T07:54:26.475 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:26 smithi053 ceph-mon[36437]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T07:54:26.475 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:26.475 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:26.475 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:26 smithi053 ceph-mon[36437]: from='osd.11 ' entity='osd.11' 2024-04-17T07:54:27.201 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:27 smithi046 ceph-mon[32038]: purged_snaps scrub starts 2024-04-17T07:54:27.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:27 smithi046 ceph-mon[32038]: purged_snaps scrub ok 2024-04-17T07:54:27.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:27 smithi046 ceph-mon[32038]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:27.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:27.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:27 smithi046 ceph-mon[32038]: osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] boot 2024-04-17T07:54:27.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:27 smithi046 ceph-mon[32038]: osdmap e80: 12 total, 12 up, 12 in 2024-04-17T07:54:27.202 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 -- 172.21.15.46:0/3729303228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 msgr2=0x7f2420075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 --2- 172.21.15.46:0/3729303228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 0x7f2420075930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f24100098f0 tx=0x7f2410009c00 comp rx=0 tx=0).stop 2024-04-17T07:54:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 -- 172.21.15.46:0/3729303228 shutdown_connections 2024-04-17T07:54:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 --2- 172.21.15.46:0/3729303228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242010a990 0x7f242010ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 --2- 172.21.15.46:0/3729303228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2420076a80 0x7f2420074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 --2- 172.21.15.46:0/3729303228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 0x7f2420075930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 -- 172.21.15.46:0/3729303228 >> 172.21.15.46:0/3729303228 conn(0x7f24200fe020 msgr2=0x7f24201004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 -- 172.21.15.46:0/3729303228 shutdown_connections 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.370+0000 7f2428800700 1 -- 172.21.15.46:0/3729303228 wait complete. 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.371+0000 7f2428800700 1 Processor -- start 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.371+0000 7f2428800700 1 -- start start 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.371+0000 7f2428800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 0x7f242019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.371+0000 7f2428800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2420076a80 0x7f242019b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2428800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242010a990 0x7f24201a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2428800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f24201108f0 con 0x7f2420076a80 2024-04-17T07:54:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2428800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2420110770 con 0x7f24200754b0 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2428800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2420110a70 con 0x7f242010a990 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2420076a80 0x7f242019b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2426800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242010a990 0x7f24201a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2426800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242010a990 0x7f24201a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:32944/0 (socket says 172.21.15.46:32944) 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2426800700 1 -- 172.21.15.46:0/2337865804 learned_addr learned my addr 172.21.15.46:0/2337865804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425e00700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 0x7f242019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425400700 1 -- 172.21.15.46:0/2337865804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242010a990 msgr2=0x7f24201a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425400700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242010a990 0x7f24201a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425400700 1 -- 172.21.15.46:0/2337865804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 msgr2=0x7f242019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425400700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 0x7f242019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425400700 1 -- 172.21.15.46:0/2337865804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24201a2c00 con 0x7f2420076a80 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2426800700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242010a990 0x7f24201a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:54:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.372+0000 7f2425e00700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 0x7f242019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.373+0000 7f2425400700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2420076a80 0x7f242019b530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2410000c00 tx=0x7f2410005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.373+0000 7f241aa00700 1 -- 172.21.15.46:0/2337865804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241000f950 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.373+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24201a2e90 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.373+0000 7f241aa00700 1 -- 172.21.15.46:0/2337865804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410005650 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.373+0000 7f241aa00700 1 -- 172.21.15.46:0/2337865804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410003d90 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.373+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24201a3280 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.375+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2420069740 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.379+0000 7f241aa00700 1 -- 172.21.15.46:0/2337865804 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f241000bbb0 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.381+0000 7f241aa00700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f24080a3e60 0x7f24080a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.382+0000 7f241aa00700 1 -- 172.21.15.46:0/2337865804 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f24100d0c10 con 0x7f2420076a80 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.382+0000 7f2425e00700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f24080a3e60 0x7f24080a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.382+0000 7f2425e00700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f24080a3e60 0x7f24080a6320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2420076e60 tx=0x7f2414009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.382+0000 7f241aa00700 1 -- 172.21.15.46:0/2337865804 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2410013030 con 0x7f2420076a80 2024-04-17T07:54:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:27 smithi053 ceph-mon[36437]: purged_snaps scrub starts 2024-04-17T07:54:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:27 smithi053 ceph-mon[36437]: purged_snaps scrub ok 2024-04-17T07:54:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:27 smithi053 ceph-mon[36437]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:27 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:27 smithi053 ceph-mon[36437]: osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] boot 2024-04-17T07:54:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:27 smithi053 ceph-mon[36437]: osdmap e80: 12 total, 12 up, 12 in 2024-04-17T07:54:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:27 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:27.498 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:27 smithi062 ceph-mon[36507]: purged_snaps scrub starts 2024-04-17T07:54:27.498 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:27 smithi062 ceph-mon[36507]: purged_snaps scrub ok 2024-04-17T07:54:27.498 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:27 smithi062 ceph-mon[36507]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:54:27.498 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:27 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:27.499 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:27 smithi062 ceph-mon[36507]: osd.11 [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] boot 2024-04-17T07:54:27.499 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:27 smithi062 ceph-mon[36507]: osdmap e80: 12 total, 12 up, 12 in 2024-04-17T07:54:27.499 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:27 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:54:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.621+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2420063660 con 0x7f2420076a80 2024-04-17T07:54:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.622+0000 7f241aa00700 1 -- 172.21.15.46:0/2337865804 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f241001fe50 con 0x7f2420076a80 2024-04-17T07:54:27.623 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f24080a3e60 msgr2=0x7f24080a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f24080a3e60 0x7f24080a6320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2420076e60 tx=0x7f2414009260 comp rx=0 tx=0).stop 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2420076a80 msgr2=0x7f242019b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2420076a80 0x7f242019b530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2410000c00 tx=0x7f2410005250 comp rx=0 tx=0).stop 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 shutdown_connections 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f24080a3e60 0x7f24080a6320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242010a990 0x7f24201a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200754b0 0x7f242019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 --2- 172.21.15.46:0/2337865804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2420076a80 0x7f242019b530 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 >> 172.21.15.46:0/2337865804 conn(0x7f24200fe020 msgr2=0x7f2420106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 shutdown_connections 2024-04-17T07:54:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:27.625+0000 7f2428800700 1 -- 172.21.15.46:0/2337865804 wait complete. 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2337865804' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:28.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:28.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2337865804' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:28.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:28.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.441 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1713340466,"num_in_osds":12,"osd_in_since":1713340446,"num_remapped_pgs":0} 2024-04-17T07:54:28.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T07:54:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2337865804' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T07:54:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T07:54:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:28.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:54:29.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:29 smithi046 ceph-mon[32038]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:29.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:29 smithi046 ceph-mon[32038]: Detected new or changed devices on smithi062 2024-04-17T07:54:29.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:29 smithi046 ceph-mon[32038]: Adjusting osd_memory_target on smithi062 to 4270M 2024-04-17T07:54:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:29 smithi053 ceph-mon[36437]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:29 smithi053 ceph-mon[36437]: Detected new or changed devices on smithi062 2024-04-17T07:54:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:29 smithi053 ceph-mon[36437]: Adjusting osd_memory_target on smithi062 to 4270M 2024-04-17T07:54:29.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:29 smithi062 ceph-mon[36507]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:29.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:29 smithi062 ceph-mon[36507]: Detected new or changed devices on smithi062 2024-04-17T07:54:29.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:29 smithi062 ceph-mon[36507]: Adjusting osd_memory_target on smithi062 to 4270M 2024-04-17T07:54:31.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 -- 172.21.15.46:0/1520212869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 msgr2=0x7f8184103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:31.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 --2- 172.21.15.46:0/1520212869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 0x7f8184103fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8174009990 tx=0x7f8174009ca0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 -- 172.21.15.46:0/1520212869 shutdown_connections 2024-04-17T07:54:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 --2- 172.21.15.46:0/1520212869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818410cfa0 0x7f818410f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 --2- 172.21.15.46:0/1520212869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 0x7f8184103fa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 --2- 172.21.15.46:0/1520212869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81841044e0 0x7f818410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 -- 172.21.15.46:0/1520212869 >> 172.21.15.46:0/1520212869 conn(0x7f81840fd370 msgr2=0x7f81840ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 -- 172.21.15.46:0/1520212869 shutdown_connections 2024-04-17T07:54:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.309+0000 7f8188a00700 1 -- 172.21.15.46:0/1520212869 wait complete. 2024-04-17T07:54:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.310+0000 7f8188a00700 1 Processor -- start 2024-04-17T07:54:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8188a00700 1 -- start start 2024-04-17T07:54:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8188a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 0x7f8184101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8188a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81841044e0 0x7f8184101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8188a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818410cfa0 0x7f81841022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8188a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8184112d00 con 0x7f818410cfa0 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8188a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8184112b80 con 0x7f81841044e0 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8188a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8184112e80 con 0x7f8184103bc0 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8183e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818410cfa0 0x7f81841022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8182a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81841044e0 0x7f8184101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8183e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818410cfa0 0x7f81841022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58804/0 (socket says 172.21.15.46:58804) 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8183e00700 1 -- 172.21.15.46:0/2226546667 learned_addr learned my addr 172.21.15.46:0/2226546667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8182a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81841044e0 0x7f8184101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47140/0 (socket says 172.21.15.46:47140) 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.311+0000 7f8183400700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 0x7f8184101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f8182a00700 1 -- 172.21.15.46:0/2226546667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 msgr2=0x7f8184101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f8182a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 0x7f8184101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f8182a00700 1 -- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818410cfa0 msgr2=0x7f81841022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f8182a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818410cfa0 0x7f81841022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f8182a00700 1 -- 172.21.15.46:0/2226546667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81841a76c0 con 0x7f81841044e0 2024-04-17T07:54:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f8182a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81841044e0 0x7f8184101d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f817000e9c0 tx=0x7f817000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f817be00700 1 -- 172.21.15.46:0/2226546667 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817000c9c0 con 0x7f81841044e0 2024-04-17T07:54:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f817be00700 1 -- 172.21.15.46:0/2226546667 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817000cb60 con 0x7f81841044e0 2024-04-17T07:54:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.312+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81841a79b0 con 0x7f81841044e0 2024-04-17T07:54:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.313+0000 7f817be00700 1 -- 172.21.15.46:0/2226546667 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170018680 con 0x7f81841044e0 2024-04-17T07:54:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.313+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81841a7ef0 con 0x7f81841044e0 2024-04-17T07:54:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.314+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8184069740 con 0x7f81841044e0 2024-04-17T07:54:31.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.316+0000 7f817be00700 1 -- 172.21.15.46:0/2226546667 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8170010430 con 0x7f81841044e0 2024-04-17T07:54:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.317+0000 7f817be00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81680a3c40 0x7f81680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:31.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.317+0000 7f817be00700 1 -- 172.21.15.46:0/2226546667 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f81700d00b0 con 0x7f81841044e0 2024-04-17T07:54:31.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.317+0000 7f8183400700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81680a3c40 0x7f81680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:31.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.318+0000 7f8183400700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81680a3c40 0x7f81680a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f817400b2e0 tx=0x7f8174005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:31.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.322+0000 7f817be00700 1 -- 172.21.15.46:0/2226546667 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f817009ce90 con 0x7f81841044e0 2024-04-17T07:54:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:31 smithi053 ceph-mon[36437]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:31 smithi062 ceph-mon[36507]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.552+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f818402df90 con 0x7f81841044e0 2024-04-17T07:54:31.554 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:31 smithi046 ceph-mon[32038]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.553+0000 7f817be00700 1 -- 172.21.15.46:0/2226546667 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18319 (secure 0 0 0) 0x7f817009c790 con 0x7f81841044e0 2024-04-17T07:54:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:31.555 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":81,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:54:27.818161+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81680a3c40 msgr2=0x7f81680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81680a3c40 0x7f81680a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f817400b2e0 tx=0x7f8174005fd0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81841044e0 msgr2=0x7f8184101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81841044e0 0x7f8184101d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f817000e9c0 tx=0x7f817000ecd0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 shutdown_connections 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81680a3c40 0x7f81680a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f818410cfa0 0x7f81841022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8184103bc0 0x7f8184101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 --2- 172.21.15.46:0/2226546667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81841044e0 0x7f8184101d60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:31.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.557+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 >> 172.21.15.46:0/2226546667 conn(0x7f81840fd370 msgr2=0x7f81840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:31.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.558+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 shutdown_connections 2024-04-17T07:54:31.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:31.558+0000 7f8188a00700 1 -- 172.21.15.46:0/2226546667 wait complete. 2024-04-17T07:54:32.315 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:32 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2226546667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:54:32.398 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T07:54:32.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-17T07:54:32.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:32 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2226546667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:54:32.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:32 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2226546667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:54:32.875 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:54:33.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:33 smithi046 ceph-mon[32038]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:33.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:33 smithi053 ceph-mon[36437]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:33.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:33 smithi062 ceph-mon[36507]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 -- 172.21.15.46:0/4076897708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964109c90 msgr2=0x7fe96410dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 --2- 172.21.15.46:0/4076897708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964109c90 0x7fe96410dce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe950009920 tx=0x7fe950009c30 comp rx=0 tx=0).stop 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 -- 172.21.15.46:0/4076897708 shutdown_connections 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 --2- 172.21.15.46:0/4076897708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe96410e3c0 0x7fe964115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 --2- 172.21.15.46:0/4076897708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9641092e0 0x7fe9641096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 --2- 172.21.15.46:0/4076897708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964109c90 0x7fe96410dce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 -- 172.21.15.46:0/4076897708 >> 172.21.15.46:0/4076897708 conn(0x7fe964076220 msgr2=0x7fe964107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 -- 172.21.15.46:0/4076897708 shutdown_connections 2024-04-17T07:54:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.090+0000 7fe96c000700 1 -- 172.21.15.46:0/4076897708 wait complete. 2024-04-17T07:54:35.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.091+0000 7fe96c000700 1 Processor -- start 2024-04-17T07:54:35.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.091+0000 7fe96c000700 1 -- start start 2024-04-17T07:54:35.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe96c000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9641092e0 0x7fe9641a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe96c000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe964109c90 0x7fe9641a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe96c000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96410e3c0 0x7fe96419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe96c000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641193f0 con 0x7fe96410e3c0 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe96c000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe964119270 con 0x7fe964109c90 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe96c000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe964119570 con 0x7fe9641092e0 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe969600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9641092e0 0x7fe9641a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe968c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe964109c90 0x7fe9641a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe969600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9641092e0 0x7fe9641a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:52150/0 (socket says 172.21.15.46:52150) 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe96a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96410e3c0 0x7fe96419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe969600700 1 -- 172.21.15.46:0/4123802630 learned_addr learned my addr 172.21.15.46:0/4123802630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe968c00700 1 -- 172.21.15.46:0/4123802630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9641092e0 msgr2=0x7fe9641a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe968c00700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9641092e0 0x7fe9641a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe968c00700 1 -- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96410e3c0 msgr2=0x7fe96419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe968c00700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96410e3c0 0x7fe96419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.092+0000 7fe968c00700 1 -- 172.21.15.46:0/4123802630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9641a0050 con 0x7fe964109c90 2024-04-17T07:54:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.093+0000 7fe968c00700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe964109c90 0x7fe9641a5b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe95000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.093+0000 7fe962000700 1 -- 172.21.15.46:0/4123802630 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500050c0 con 0x7fe964109c90 2024-04-17T07:54:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.093+0000 7fe962000700 1 -- 172.21.15.46:0/4123802630 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95000b560 con 0x7fe964109c90 2024-04-17T07:54:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.093+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9641a02e0 con 0x7fe964109c90 2024-04-17T07:54:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.093+0000 7fe962000700 1 -- 172.21.15.46:0/4123802630 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950003a00 con 0x7fe964109c90 2024-04-17T07:54:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.093+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641ac400 con 0x7fe964109c90 2024-04-17T07:54:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.094+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96410a7c0 con 0x7fe964109c90 2024-04-17T07:54:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.097+0000 7fe962000700 1 -- 172.21.15.46:0/4123802630 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe950029060 con 0x7fe964109c90 2024-04-17T07:54:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.098+0000 7fe962000700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe94c0a3cc0 0x7fe94c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.098+0000 7fe962000700 1 -- 172.21.15.46:0/4123802630 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe9500d0e90 con 0x7fe964109c90 2024-04-17T07:54:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.098+0000 7fe969600700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe94c0a3cc0 0x7fe94c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.099+0000 7fe969600700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe94c0a3cc0 0x7fe94c0a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe958005950 tx=0x7fe958009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.100+0000 7fe962000700 1 -- 172.21.15.46:0/4123802630 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe950013030 con 0x7fe964109c90 2024-04-17T07:54:35.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.332+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe96402df90 con 0x7fe964109c90 2024-04-17T07:54:35.333 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:35 smithi046 ceph-mon[32038]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:35.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.332+0000 7fe962000700 1 -- 172.21.15.46:0/4123802630 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fe95009dd00 con 0x7fe964109c90 2024-04-17T07:54:35.334 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:54:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe94c0a3cc0 msgr2=0x7fe94c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe94c0a3cc0 0x7fe94c0a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe958005950 tx=0x7fe958009310 comp rx=0 tx=0).stop 2024-04-17T07:54:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe964109c90 msgr2=0x7fe9641a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe964109c90 0x7fe9641a5b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe95000fa70 comp rx=0 tx=0).stop 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 shutdown_connections 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe94c0a3cc0 0x7fe94c0a6180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe96410e3c0 0x7fe96419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9641092e0 0x7fe9641a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 --2- 172.21.15.46:0/4123802630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe964109c90 0x7fe9641a5b90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 >> 172.21.15.46:0/4123802630 conn(0x7fe964076220 msgr2=0x7fe964077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 shutdown_connections 2024-04-17T07:54:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:35.336+0000 7fe96c000700 1 -- 172.21.15.46:0/4123802630 wait complete. 2024-04-17T07:54:35.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:35 smithi053 ceph-mon[36437]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:35.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:35 smithi062 ceph-mon[36507]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.a on smithi046 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.d on smithi046 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.g on smithi046 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.j on smithi046 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.b on smithi053 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.e on smithi053 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.h on smithi053 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.k on smithi053 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.c on smithi062 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.f on smithi062 2024-04-17T07:54:36.287 INFO:tasks.cephadm:Adding mds.i on smithi062 2024-04-17T07:54:36.288 INFO:tasks.cephadm:Adding mds.l on smithi062 2024-04-17T07:54:36.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph orch apply mds all '12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l' 2024-04-17T07:54:36.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:36 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4123802630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:54:36.540 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:54:36.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:36 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4123802630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:54:36.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:36 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4123802630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:54:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:37 smithi053 ceph-mon[36437]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:37.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:37 smithi062 ceph-mon[36507]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:37.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:37 smithi046 ceph-mon[32038]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 -- 172.21.15.62:0/3933712488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8294108830 msgr2=0x7f829410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3933712488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8294108830 0x7f829410ac20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f828c00b0a0 tx=0x7f828c00b3b0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 -- 172.21.15.62:0/3933712488 shutdown_connections 2024-04-17T07:54:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3933712488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8294108830 0x7f829410ac20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3933712488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82941035d0 0x7f82941059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3933712488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8294105f00 0x7f82941082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 -- 172.21.15.62:0/3933712488 >> 172.21.15.62:0/3933712488 conn(0x7f82940fcf40 msgr2=0x7f82940ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 -- 172.21.15.62:0/3933712488 shutdown_connections 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.178+0000 7f829b9e3700 1 -- 172.21.15.62:0/3933712488 wait complete. 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 Processor -- start 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 -- start start 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82941035d0 0x7f8294198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8294105f00 0x7f8294199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8294108830 0x7f82941a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f829410e750 con 0x7f8294108830 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f829410e5d0 con 0x7f8294105f00 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829b9e3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f829410e8d0 con 0x7f82941035d0 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829977f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82941035d0 0x7f8294198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829977f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82941035d0 0x7f8294198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48526/0 (socket says 172.21.15.62:48526) 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.179+0000 7f829977f700 1 -- 172.21.15.62:0/3974137345 learned_addr learned my addr 172.21.15.62:0/3974137345 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f8298f7e700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8294105f00 0x7f8294199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829977f700 1 -- 172.21.15.62:0/3974137345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8294105f00 msgr2=0x7f8294199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829977f700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8294105f00 0x7f8294199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829977f700 1 -- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8294108830 msgr2=0x7f82941a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829977f700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8294108830 0x7f82941a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829977f700 1 -- 172.21.15.62:0/3974137345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82941a0940 con 0x7f82941035d0 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f8298f7e700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8294105f00 0x7f8294199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829977f700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82941035d0 0x7f8294198cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f828800b530 tx=0x7f828800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f82867fc700 1 -- 172.21.15.62:0/3974137345 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288015020 con 0x7f82941035d0 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f82867fc700 1 -- 172.21.15.62:0/3974137345 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8288011d10 con 0x7f82941035d0 2024-04-17T07:54:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82941a0c30 con 0x7f82941035d0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f82867fc700 1 -- 172.21.15.62:0/3974137345 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288010e50 con 0x7f82941035d0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.180+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82941a11c0 con 0x7f82941035d0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.181+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8294069740 con 0x7f82941035d0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.182+0000 7f82867fc700 1 -- 172.21.15.62:0/3974137345 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8288018020 con 0x7f82941035d0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.183+0000 7f82867fc700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82800a3e60 0x7f82800a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.183+0000 7f82867fc700 1 -- 172.21.15.62:0/3974137345 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f82880cf740 con 0x7f82941035d0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.183+0000 7f8298f7e700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82800a3e60 0x7f82800a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.184+0000 7f8298f7e700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82800a3e60 0x7f82800a6320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8290007900 tx=0x7f8290008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.188+0000 7f82867fc700 1 -- 172.21.15.62:0/3974137345 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f828801f020 con 0x7f82941035d0 2024-04-17T07:54:38.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.409+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l", "target": ["mon-mgr", ""]}) -- 0x7f8294063090 con 0x7f82800a3e60 2024-04-17T07:54:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.425+0000 7f82867fc700 1 -- 172.21.15.62:0/3974137345 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8294063090 con 0x7f82800a3e60 2024-04-17T07:54:38.425 INFO:teuthology.orchestra.run.smithi062.stdout:Scheduled mds.all update... 2024-04-17T07:54:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82800a3e60 msgr2=0x7f82800a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82800a3e60 0x7f82800a6320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8290007900 tx=0x7f8290008040 comp rx=0 tx=0).stop 2024-04-17T07:54:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82941035d0 msgr2=0x7f8294198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82941035d0 0x7f8294198cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f828800b530 tx=0x7f828800b8f0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 shutdown_connections 2024-04-17T07:54:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82800a3e60 0x7f82800a6320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8294108830 0x7f82941a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82941035d0 0x7f8294198cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 --2- 172.21.15.62:0/3974137345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8294105f00 0x7f8294199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 >> 172.21.15.62:0/3974137345 conn(0x7f82940fcf40 msgr2=0x7f8294103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 shutdown_connections 2024-04-17T07:54:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-17T07:54:38.428+0000 7f829b9e3700 1 -- 172.21.15.62:0/3974137345 wait complete. 2024-04-17T07:54:39.243 DEBUG:teuthology.orchestra.run.smithi046:mds.a> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.a.service 2024-04-17T07:54:39.249 DEBUG:teuthology.orchestra.run.smithi046:mds.d> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.d.service 2024-04-17T07:54:39.295 DEBUG:teuthology.orchestra.run.smithi046:mds.g> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.g.service 2024-04-17T07:54:39.341 DEBUG:teuthology.orchestra.run.smithi046:mds.j> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.j.service 2024-04-17T07:54:39.386 DEBUG:teuthology.orchestra.run.smithi053:mds.b> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.b.service 2024-04-17T07:54:39.389 DEBUG:teuthology.orchestra.run.smithi053:mds.e> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.e.service 2024-04-17T07:54:39.419 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:39 smithi053 ceph-mon[36437]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:39.419 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:39.419 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:39 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:39.420 DEBUG:teuthology.orchestra.run.smithi053:mds.h> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.h.service 2024-04-17T07:54:39.447 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:39 smithi046 ceph-mon[32038]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:39.448 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:39.448 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:39 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:39.463 DEBUG:teuthology.orchestra.run.smithi053:mds.k> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.k.service 2024-04-17T07:54:39.507 DEBUG:teuthology.orchestra.run.smithi062:mds.c> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.c.service 2024-04-17T07:54:39.510 DEBUG:teuthology.orchestra.run.smithi062:mds.f> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.f.service 2024-04-17T07:54:39.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:39 smithi062 ceph-mon[36507]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:39.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:39.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:39 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:54:39.537 DEBUG:teuthology.orchestra.run.smithi062:mds.i> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.i.service 2024-04-17T07:54:39.580 DEBUG:teuthology.orchestra.run.smithi062:mds.l> sudo journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.l.service 2024-04-17T07:54:39.623 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-17T07:54:39.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:54:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 -- 172.21.15.46:0/2590683086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c10a190 msgr2=0x7f319c110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 --2- 172.21.15.46:0/2590683086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c10a190 0x7f319c110a20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3194009900 tx=0x7f319402bc30 comp rx=0 tx=0).stop 2024-04-17T07:54:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 -- 172.21.15.46:0/2590683086 shutdown_connections 2024-04-17T07:54:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 --2- 172.21.15.46:0/2590683086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c10a190 0x7f319c110a20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 --2- 172.21.15.46:0/2590683086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c1097d0 0x7f319c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 --2- 172.21.15.46:0/2590683086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f319c1085e0 0x7f319c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 -- 172.21.15.46:0/2590683086 >> 172.21.15.46:0/2590683086 conn(0x7f319c0775b0 msgr2=0x7f319c0799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 -- 172.21.15.46:0/2590683086 shutdown_connections 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.861+0000 7f31a1600640 1 -- 172.21.15.46:0/2590683086 wait complete. 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.862+0000 7f31a1600640 1 Processor -- start 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.862+0000 7f31a1600640 1 -- start start 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f31a1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f319c1085e0 0x7f319c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f31a1600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c1097d0 0x7f319c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f31a1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c10a190 0x7f319c1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f31a1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f319c114270 con 0x7f319c1085e0 2024-04-17T07:54:39.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f31a1600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1140f0 con 0x7f319c1097d0 2024-04-17T07:54:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f31a1600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1143f0 con 0x7f319c10a190 2024-04-17T07:54:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f3192000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c1097d0 0x7f319c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f319b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c10a190 0x7f319c1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f319b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c10a190 0x7f319c1a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39882/0 (socket says 172.21.15.46:39882) 2024-04-17T07:54:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f319b400640 1 -- 172.21.15.46:0/1268291897 learned_addr learned my addr 172.21.15.46:0/1268291897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f3192000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c1097d0 0x7f319c19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56126/0 (socket says 172.21.15.46:56126) 2024-04-17T07:54:39.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f319aa00640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f319c1085e0 0x7f319c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:39.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f319b400640 1 -- 172.21.15.46:0/1268291897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c1097d0 msgr2=0x7f319c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:39.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.863+0000 7f319b400640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c1097d0 0x7f319c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:39.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f319b400640 1 -- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f319c1085e0 msgr2=0x7f319c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:39.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f319b400640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f319c1085e0 0x7f319c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:39.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f319b400640 1 -- 172.21.15.46:0/1268291897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1a64a0 con 0x7f319c10a190 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f3192000640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c1097d0 0x7f319c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f319b400640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c10a190 0x7f319c1a5e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3194036040 tx=0x7f3194005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f3193e00640 1 -- 172.21.15.46:0/1268291897 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194043020 con 0x7f319c10a190 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f3193e00640 1 -- 172.21.15.46:0/1268291897 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31940054e0 con 0x7f319c10a190 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1a6730 con 0x7f319c10a190 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f3193e00640 1 -- 172.21.15.46:0/1268291897 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194003c50 con 0x7f319c10a190 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.864+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a6cd0 con 0x7f319c10a190 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.865+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c10f8a0 con 0x7f319c10a190 2024-04-17T07:54:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.867+0000 7f3193e00640 1 -- 172.21.15.46:0/1268291897 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3194046020 con 0x7f319c10a190 2024-04-17T07:54:39.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.869+0000 7f3193e00640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31800a3c80 0x7f31800a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.869+0000 7f3193e00640 1 -- 172.21.15.46:0/1268291897 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f31940f2620 con 0x7f319c10a190 2024-04-17T07:54:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.869+0000 7f319aa00640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31800a3c80 0x7f31800a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.870+0000 7f319aa00640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31800a3c80 0x7f31800a6140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f319c109630 tx=0x7f318c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:39.871+0000 7f3193e00640 1 -- 172.21.15.46:0/1268291897 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f319404d020 con 0x7f319c10a190 2024-04-17T07:54:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.081+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f319c02d7a0 con 0x7f319c10a190 2024-04-17T07:54:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.083+0000 7f3193e00640 1 -- 172.21.15.46:0/1268291897 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18319 (secure 0 0 0) 0x7f31940bf400 con 0x7f319c10a190 2024-04-17T07:54:40.083 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:40.084 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":81,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:54:27.818161+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31800a3c80 msgr2=0x7f31800a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31800a3c80 0x7f31800a6140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f319c109630 tx=0x7f318c009500 comp rx=0 tx=0).stop 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c10a190 msgr2=0x7f319c1a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c10a190 0x7f319c1a5e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3194036040 tx=0x7f3194005a20 comp rx=0 tx=0).stop 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 shutdown_connections 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31800a3c80 0x7f31800a6140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f319c10a190 0x7f319c1a5e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c1097d0 0x7f319c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 --2- 172.21.15.46:0/1268291897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f319c1085e0 0x7f319c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 >> 172.21.15.46:0/1268291897 conn(0x7f319c0775b0 msgr2=0x7f319c10ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 shutdown_connections 2024-04-17T07:54:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.085+0000 7f31a1600640 1 -- 172.21.15.46:0/1268291897 wait complete. 2024-04-17T07:54:40.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T07:54:40.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:54:40.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.346+0000 7f1acc600640 1 -- 172.21.15.46:0/552147112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41097d0 msgr2=0x7f1ac4109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 --2- 172.21.15.46:0/552147112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41097d0 0x7f1ac4109c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009900 tx=0x7f1ab002bc30 comp rx=0 tx=0).stop 2024-04-17T07:54:40.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 -- 172.21.15.46:0/552147112 shutdown_connections 2024-04-17T07:54:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 --2- 172.21.15.46:0/552147112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac410a190 0x7f1ac4110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 --2- 172.21.15.46:0/552147112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41097d0 0x7f1ac4109c50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 --2- 172.21.15.46:0/552147112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac41085e0 0x7f1ac4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 -- 172.21.15.46:0/552147112 >> 172.21.15.46:0/552147112 conn(0x7f1ac40775b0 msgr2=0x7f1ac40799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 -- 172.21.15.46:0/552147112 shutdown_connections 2024-04-17T07:54:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.347+0000 7f1acc600640 1 -- 172.21.15.46:0/552147112 wait complete. 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 Processor -- start 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 -- start start 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41085e0 0x7f1ac419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac41097d0 0x7f1ac419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac410a190 0x7f1ac41a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac41142a0 con 0x7f1ac410a190 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4114120 con 0x7f1ac41085e0 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.348+0000 7f1acc600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4114420 con 0x7f1ac41097d0 2024-04-17T07:54:40.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac41097d0 0x7f1ac419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac41097d0 0x7f1ac419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39906/0 (socket says 172.21.15.46:39906) 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 -- 172.21.15.46:0/3664061889 learned_addr learned my addr 172.21.15.46:0/3664061889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1aca600640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac410a190 0x7f1ac41a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9c00640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41085e0 0x7f1ac419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 -- 172.21.15.46:0/3664061889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41085e0 msgr2=0x7f1ac419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41085e0 0x7f1ac419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 -- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac410a190 msgr2=0x7f1ac41a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac410a190 0x7f1ac41a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9200640 1 -- 172.21.15.46:0/3664061889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41a64d0 con 0x7f1ac41097d0 2024-04-17T07:54:40.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.349+0000 7f1ac9c00640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41085e0 0x7f1ac419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:54:40.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.350+0000 7f1ac9200640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac41097d0 0x7f1ac419edf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0000c00 tx=0x7f1ab002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:40.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.350+0000 7f1ac2000640 1 -- 172.21.15.46:0/3664061889 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0043020 con 0x7f1ac41097d0 2024-04-17T07:54:40.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.350+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41a6760 con 0x7f1ac41097d0 2024-04-17T07:54:40.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.350+0000 7f1ac2000640 1 -- 172.21.15.46:0/3664061889 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab00054e0 con 0x7f1ac41097d0 2024-04-17T07:54:40.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.350+0000 7f1ac2000640 1 -- 172.21.15.46:0/3664061889 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0003bd0 con 0x7f1ac41097d0 2024-04-17T07:54:40.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.350+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41a6cb0 con 0x7f1ac41097d0 2024-04-17T07:54:40.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.351+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac410f900 con 0x7f1ac41097d0 2024-04-17T07:54:40.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.353+0000 7f1ac2000640 1 -- 172.21.15.46:0/3664061889 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ab0046020 con 0x7f1ac41097d0 2024-04-17T07:54:40.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.354+0000 7f1ac2000640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1aac0a3bb0 0x7f1aac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.355+0000 7f1ac2000640 1 -- 172.21.15.46:0/3664061889 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1ab00f2060 con 0x7f1ac41097d0 2024-04-17T07:54:40.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.355+0000 7f1ac9c00640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1aac0a3bb0 0x7f1aac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.355+0000 7f1ac9c00640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1aac0a3bb0 0x7f1aac0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4109630 tx=0x7f1ab8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:40.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.360+0000 7f1ac2000640 1 -- 172.21.15.46:0/3664061889 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1ab004d020 con 0x7f1ac41097d0 2024-04-17T07:54:40.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l;count:12 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:40.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:40 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1268291897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:54:40.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:54:40.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l;count:12 2024-04-17T07:54:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:40.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:40 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1268291897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:54:40.499 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:54:40.499 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l;count:12 2024-04-17T07:54:40.499 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:40.499 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:54:40.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:40.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:40.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:40.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:40.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:40 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1268291897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:54:40.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.563+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ac4062b60 con 0x7f1ac41097d0 2024-04-17T07:54:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.564+0000 7f1ac2000640 1 -- 172.21.15.46:0/3664061889 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f1ab00bee40 con 0x7f1ac41097d0 2024-04-17T07:54:40.564 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:54:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.566+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1aac0a3bb0 msgr2=0x7f1aac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.566+0000 7f1acc600640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1aac0a3bb0 0x7f1aac0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4109630 tx=0x7f1ab8009500 comp rx=0 tx=0).stop 2024-04-17T07:54:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.566+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac41097d0 msgr2=0x7f1ac419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.566+0000 7f1acc600640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac41097d0 0x7f1ac419edf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0000c00 tx=0x7f1ab002c650 comp rx=0 tx=0).stop 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 shutdown_connections 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1aac0a3bb0 0x7f1aac0a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac410a190 0x7f1ac41a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ac41097d0 0x7f1ac419edf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 --2- 172.21.15.46:0/3664061889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ac41085e0 0x7f1ac419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 >> 172.21.15.46:0/3664061889 conn(0x7f1ac40775b0 msgr2=0x7f1ac410cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 shutdown_connections 2024-04-17T07:54:40.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.567+0000 7f1acc600640 1 -- 172.21.15.46:0/3664061889 wait complete. 2024-04-17T07:54:40.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:40.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 -- 172.21.15.46:0/2284316727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c1069a0 msgr2=0x7fa80c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 --2- 172.21.15.46:0/2284316727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c1069a0 0x7fa80c108e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa804009ca0 tx=0x7fa80402c100 comp rx=0 tx=0).stop 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 -- 172.21.15.46:0/2284316727 shutdown_connections 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 --2- 172.21.15.46:0/2284316727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c1069a0 0x7fa80c108e30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 --2- 172.21.15.46:0/2284316727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c103f90 0x7fa80c106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 --2- 172.21.15.46:0/2284316727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa80c101660 0x7fa80c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 -- 172.21.15.46:0/2284316727 >> 172.21.15.46:0/2284316727 conn(0x7fa80c0fb660 msgr2=0x7fa80c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 -- 172.21.15.46:0/2284316727 shutdown_connections 2024-04-17T07:54:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.814+0000 7fa810c00640 1 -- 172.21.15.46:0/2284316727 wait complete. 2024-04-17T07:54:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 Processor -- start 2024-04-17T07:54:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 -- start start 2024-04-17T07:54:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c101660 0x7fa80c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c103f90 0x7fa80c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa80c1069a0 0x7fa80c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c10db40 con 0x7fa80c101660 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c10d9c0 con 0x7fa80c1069a0 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.815+0000 7fa810c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c10dcc0 con 0x7fa80c103f90 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c103f90 0x7fa80c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c103f90 0x7fa80c19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39926/0 (socket says 172.21.15.46:39926) 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 -- 172.21.15.46:0/4223470755 learned_addr learned my addr 172.21.15.46:0/4223470755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa80b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c101660 0x7fa80c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa80be00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa80c1069a0 0x7fa80c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 -- 172.21.15.46:0/4223470755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa80c1069a0 msgr2=0x7fa80c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa80c1069a0 0x7fa80c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 -- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c101660 msgr2=0x7fa80c19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c101660 0x7fa80c19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.816+0000 7fa803e00640 1 -- 172.21.15.46:0/4223470755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c1a1fd0 con 0x7fa80c103f90 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.817+0000 7fa803e00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c103f90 0x7fa80c19a8f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400e9f0 tx=0x7fa7f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.817+0000 7fa808c00640 1 -- 172.21.15.46:0/4223470755 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400cd80 con 0x7fa80c103f90 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.817+0000 7fa808c00640 1 -- 172.21.15.46:0/4223470755 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f4010910 con 0x7fa80c103f90 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.817+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c1a22c0 con 0x7fa80c103f90 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.817+0000 7fa808c00640 1 -- 172.21.15.46:0/4223470755 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4018990 con 0x7fa80c103f90 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.817+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c1a2860 con 0x7fa80c103f90 2024-04-17T07:54:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.818+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c108490 con 0x7fa80c103f90 2024-04-17T07:54:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.821+0000 7fa808c00640 1 -- 172.21.15.46:0/4223470755 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa7f4010ac0 con 0x7fa80c103f90 2024-04-17T07:54:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.821+0000 7fa808c00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa7f00a3c20 0x7fa7f00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.822+0000 7fa808c00640 1 -- 172.21.15.46:0/4223470755 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa7f40d0270 con 0x7fa80c103f90 2024-04-17T07:54:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.822+0000 7fa80b400640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa7f00a3c20 0x7fa7f00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.823+0000 7fa80b400640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa7f00a3c20 0x7fa7f00a60e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa80c100170 tx=0x7fa7fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:40.826+0000 7fa808c00640 1 -- 172.21.15.46:0/4223470755 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa7f409ca00 con 0x7fa80c103f90 2024-04-17T07:54:41.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.073+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa80c02d5d0 con 0x7fa80c103f90 2024-04-17T07:54:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.075+0000 7fa808c00640 1 -- 172.21.15.46:0/4223470755 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa7f40a0d00 con 0x7fa80c103f90 2024-04-17T07:54:41.075 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:41.075 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-17T07:54:41.076 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 1 2024-04-17T07:54:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.077+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa7f00a3c20 msgr2=0x7fa7f00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.077+0000 7fa810c00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa7f00a3c20 0x7fa7f00a60e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa80c100170 tx=0x7fa7fc005f90 comp rx=0 tx=0).stop 2024-04-17T07:54:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.077+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c103f90 msgr2=0x7fa80c19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.077+0000 7fa810c00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c103f90 0x7fa80c19a8f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400e9f0 tx=0x7fa7f400eeb0 comp rx=0 tx=0).stop 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 shutdown_connections 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa7f00a3c20 0x7fa7f00a60e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa80c1069a0 0x7fa80c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa80c103f90 0x7fa80c19a8f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 --2- 172.21.15.46:0/4223470755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa80c101660 0x7fa80c19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 >> 172.21.15.46:0/4223470755 conn(0x7fa80c0fb660 msgr2=0x7fa80c10b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 shutdown_connections 2024-04-17T07:54:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:41.078+0000 7fa810c00640 1 -- 172.21.15.46:0/4223470755 wait complete. 2024-04-17T07:54:41.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:41 smithi046 ceph-mon[32038]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:41.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:41 smithi046 ceph-mon[32038]: Deploying daemon mds.l on smithi062 2024-04-17T07:54:41.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:41 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3664061889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:54:41.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:41 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4223470755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:41.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:41 smithi053 ceph-mon[36437]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:41.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:41 smithi053 ceph-mon[36437]: Deploying daemon mds.l on smithi062 2024-04-17T07:54:41.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:41 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3664061889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:54:41.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:41 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4223470755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:41.508 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:41 smithi062 ceph-mon[36507]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:41.508 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:41 smithi062 ceph-mon[36507]: Deploying daemon mds.l on smithi062 2024-04-17T07:54:41.508 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:41 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3664061889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:54:41.508 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:41 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4223470755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:42.257 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 systemd[1]: Starting Ceph mds.l for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:54:42.521 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 podman[67041]: 2024-04-17 07:54:42.160672611 +0000 UTC m=+0.018196859 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:54:42.521 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 podman[67041]: 2024-04-17 07:54:42.269932789 +0000 UTC m=+0.127457035 container create a56f0800ff739480d0787ada4535d865757bd6af5ac617b7071a9dc084d1bb91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-17T07:54:42.803 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 podman[67041]: 2024-04-17 07:54:42.522655966 +0000 UTC m=+0.380180212 container init a56f0800ff739480d0787ada4535d865757bd6af5ac617b7071a9dc084d1bb91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-17T07:54:42.803 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 podman[67041]: 2024-04-17 07:54:42.525489324 +0000 UTC m=+0.383013575 container start a56f0800ff739480d0787ada4535d865757bd6af5ac617b7071a9dc084d1bb91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-17T07:54:42.803 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 ceph-mds[67061]: -- 172.21.15.62:0/117028665 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55699e9cfa40 con 0x55699e9a2c00 2024-04-17T07:54:42.803 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l[67057]: starting mds.l at 2024-04-17T07:54:42.803 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 bash[67041]: a56f0800ff739480d0787ada4535d865757bd6af5ac617b7071a9dc084d1bb91 2024-04-17T07:54:42.803 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 07:54:42 smithi062 systemd[1]: Started Ceph mds.l for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:54:43.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 -- 172.21.15.46:0/1761774588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c541096a0 msgr2=0x7f1c54109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 --2- 172.21.15.46:0/1761774588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c541096a0 0x7f1c54109b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009900 tx=0x7f1c3c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:54:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 -- 172.21.15.46:0/1761774588 shutdown_connections 2024-04-17T07:54:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 --2- 172.21.15.46:0/1761774588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c5410a060 0x7f1c541108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 --2- 172.21.15.46:0/1761774588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c541096a0 0x7f1c54109b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 --2- 172.21.15.46:0/1761774588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 0x7f1c54108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 -- 172.21.15.46:0/1761774588 >> 172.21.15.46:0/1761774588 conn(0x7f1c54077450 msgr2=0x7f1c54079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 -- 172.21.15.46:0/1761774588 shutdown_connections 2024-04-17T07:54:43.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.341+0000 7f1c58c00640 1 -- 172.21.15.46:0/1761774588 wait complete. 2024-04-17T07:54:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.342+0000 7f1c58c00640 1 Processor -- start 2024-04-17T07:54:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.342+0000 7f1c58c00640 1 -- start start 2024-04-17T07:54:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c58c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 0x7f1c5419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c58c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c541096a0 0x7f1c5419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c58c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5410a060 0x7f1c541a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c58c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54114190 con 0x7f1c54108500 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c58c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54114010 con 0x7f1c5410a060 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c58c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54114310 con 0x7f1c541096a0 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c53400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 0x7f1c5419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c53400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 0x7f1c5419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57278/0 (socket says 172.21.15.46:57278) 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c53400640 1 -- 172.21.15.46:0/711424838 learned_addr learned my addr 172.21.15.46:0/711424838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c53e00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5410a060 0x7f1c541a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.343+0000 7f1c4be00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c541096a0 0x7f1c5419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c53e00640 1 -- 172.21.15.46:0/711424838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c541096a0 msgr2=0x7f1c5419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c53e00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c541096a0 0x7f1c5419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c53e00640 1 -- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 msgr2=0x7f1c5419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c53e00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 0x7f1c5419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c53e00640 1 -- 172.21.15.46:0/711424838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c541a6380 con 0x7f1c5410a060 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c53400640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 0x7f1c5419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c53e00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5410a060 0x7f1c541a5d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009fd0 tx=0x7f1c4c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c50c00640 1 -- 172.21.15.46:0/711424838 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00ce20 con 0x7f1c5410a060 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c50c00640 1 -- 172.21.15.46:0/711424838 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c4c005020 con 0x7f1c5410a060 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c50c00640 1 -- 172.21.15.46:0/711424838 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c010720 con 0x7f1c5410a060 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c541a6670 con 0x7f1c5410a060 2024-04-17T07:54:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.344+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c541a6b40 con 0x7f1c5410a060 2024-04-17T07:54:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.346+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5410f770 con 0x7f1c5410a060 2024-04-17T07:54:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.347+0000 7f1c50c00640 1 -- 172.21.15.46:0/711424838 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1c4c005540 con 0x7f1c5410a060 2024-04-17T07:54:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.349+0000 7f1c50c00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1c340a3c20 0x7f1c340a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.349+0000 7f1c50c00640 1 -- 172.21.15.46:0/711424838 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1c4c0d0680 con 0x7f1c5410a060 2024-04-17T07:54:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.349+0000 7f1c53400640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1c340a3c20 0x7f1c340a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.350+0000 7f1c53400640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1c340a3c20 0x7f1c340a60e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c440058c0 tx=0x7f1c44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.353+0000 7f1c50c00640 1 -- 172.21.15.46:0/711424838 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1c4c09d460 con 0x7f1c5410a060 2024-04-17T07:54:43.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:43.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.62:6832/1314622841,v1:172.21.15.62:6833/1314622841] up:boot 2024-04-17T07:54:43.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: fsmap 1 up:standby 2024-04-17T07:54:43.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T07:54:43.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:43.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:43.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:43 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.62:6832/1314622841,v1:172.21.15.62:6833/1314622841] up:boot 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: fsmap 1 up:standby 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:43 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:43.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.604+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c5402d5d0 con 0x7f1c5410a060 2024-04-17T07:54:43.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.604+0000 7f1c50c00640 1 -- 172.21.15.46:0/711424838 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f1c4c09cd60 con 0x7f1c5410a060 2024-04-17T07:54:43.604 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:43.604 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-17T07:54:43.604 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1c340a3c20 msgr2=0x7f1c340a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1c340a3c20 0x7f1c340a60e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c440058c0 tx=0x7f1c44005f90 comp rx=0 tx=0).stop 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5410a060 msgr2=0x7f1c541a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5410a060 0x7f1c541a5d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009fd0 tx=0x7f1c4c00ef10 comp rx=0 tx=0).stop 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 shutdown_connections 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1c340a3c20 0x7f1c340a60e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5410a060 0x7f1c541a5d10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c541096a0 0x7f1c5419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 --2- 172.21.15.46:0/711424838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c54108500 0x7f1c5419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:43.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.607+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 >> 172.21.15.46:0/711424838 conn(0x7f1c54077450 msgr2=0x7f1c5410c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.608+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 shutdown_connections 2024-04-17T07:54:43.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:43.608+0000 7f1c58c00640 1 -- 172.21.15.46:0/711424838 wait complete. 2024-04-17T07:54:43.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:43.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.62:6832/1314622841,v1:172.21.15.62:6833/1314622841] up:boot 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: fsmap 1 up:standby 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:43.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:43 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:44.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:44 smithi053 ceph-mon[36437]: Deploying daemon mds.k on smithi053 2024-04-17T07:54:44.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:44 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/711424838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:44.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:44 smithi062 ceph-mon[36507]: Deploying daemon mds.k on smithi053 2024-04-17T07:54:44.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:44 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/711424838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:44.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:44 smithi046 ceph-mon[32038]: Deploying daemon mds.k on smithi053 2024-04-17T07:54:44.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:44 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/711424838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:45.195 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 07:54:45 smithi053 podman[66427]: 2024-04-17 07:54:45.114767821 +0000 UTC m=+0.138960638 container create 236db599450973b48fb2c439d2038a568dacef853d03dcd7dcc4fc9ff13c673f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-17T07:54:45.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:45 smithi053 ceph-mon[36437]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:45.471 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 07:54:45 smithi053 podman[66427]: 2024-04-17 07:54:45.308907891 +0000 UTC m=+0.333100697 container init 236db599450973b48fb2c439d2038a568dacef853d03dcd7dcc4fc9ff13c673f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-17T07:54:45.472 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 07:54:45 smithi053 podman[66427]: 2024-04-17 07:54:45.311822128 +0000 UTC m=+0.336014927 container start 236db599450973b48fb2c439d2038a568dacef853d03dcd7dcc4fc9ff13c673f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-17T07:54:45.472 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 07:54:45 smithi053 ceph-mds[66446]: -- 172.21.15.53:0/2023605746 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fce6bcfa40 con 0x55fce6ba3000 2024-04-17T07:54:45.472 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 07:54:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k[66442]: starting mds.k at 2024-04-17T07:54:45.472 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 07:54:45 smithi053 bash[66427]: 236db599450973b48fb2c439d2038a568dacef853d03dcd7dcc4fc9ff13c673f 2024-04-17T07:54:45.472 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 07:54:45 smithi053 systemd[1]: Started Ceph mds.k for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:54:45.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:45 smithi062 ceph-mon[36507]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:45.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:45 smithi046 ceph-mon[32038]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:45.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.859+0000 7f34f1600640 1 -- 172.21.15.46:0/2404484980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec105420 msgr2=0x7f34ec1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.859+0000 7f34f1600640 1 --2- 172.21.15.46:0/2404484980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec105420 0x7f34ec1058a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009900 tx=0x7f34d802bc30 comp rx=0 tx=0).stop 2024-04-17T07:54:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.859+0000 7f34f1600640 1 -- 172.21.15.46:0/2404484980 shutdown_connections 2024-04-17T07:54:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.859+0000 7f34f1600640 1 --2- 172.21.15.46:0/2404484980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34ec105de0 0x7f34ec10c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.859+0000 7f34f1600640 1 --2- 172.21.15.46:0/2404484980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec105420 0x7f34ec1058a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.859+0000 7f34f1600640 1 --2- 172.21.15.46:0/2404484980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34ec104230 0x7f34ec104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.859+0000 7f34f1600640 1 -- 172.21.15.46:0/2404484980 >> 172.21.15.46:0/2404484980 conn(0x7f34ec0ff7e0 msgr2=0x7f34ec101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.860+0000 7f34f1600640 1 -- 172.21.15.46:0/2404484980 shutdown_connections 2024-04-17T07:54:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.860+0000 7f34f1600640 1 -- 172.21.15.46:0/2404484980 wait complete. 2024-04-17T07:54:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.861+0000 7f34f1600640 1 Processor -- start 2024-04-17T07:54:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.861+0000 7f34f1600640 1 -- start start 2024-04-17T07:54:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34f1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec104230 0x7f34ec19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34f1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34ec105420 0x7f34ec19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34f1600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34ec105de0 0x7f34ec1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34f1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec10fea0 con 0x7f34ec105420 2024-04-17T07:54:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34f1600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec10fd20 con 0x7f34ec105de0 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34f1600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec110020 con 0x7f34ec104230 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34ec105420 0x7f34ec19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec104230 0x7f34ec19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34ec105420 0x7f34ec19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57294/0 (socket says 172.21.15.46:57294) 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34ea000640 1 -- 172.21.15.46:0/3729708317 learned_addr learned my addr 172.21.15.46:0/3729708317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.862+0000 7f34eb400640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34ec105de0 0x7f34ec1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34eaa00640 1 -- 172.21.15.46:0/3729708317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34ec105de0 msgr2=0x7f34ec1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34eaa00640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34ec105de0 0x7f34ec1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34eaa00640 1 -- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34ec105420 msgr2=0x7f34ec19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34eaa00640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34ec105420 0x7f34ec19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34eaa00640 1 -- 172.21.15.46:0/3729708317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec1a20b0 con 0x7f34ec104230 2024-04-17T07:54:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34eaa00640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec104230 0x7f34ec19a490 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00e9c0 tx=0x7f34dc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34e3400640 1 -- 172.21.15.46:0/3729708317 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00ccc0 con 0x7f34ec104230 2024-04-17T07:54:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34e3400640 1 -- 172.21.15.46:0/3729708317 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34dc0035f0 con 0x7f34ec104230 2024-04-17T07:54:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec1a23a0 con 0x7f34ec104230 2024-04-17T07:54:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.863+0000 7f34e3400640 1 -- 172.21.15.46:0/3729708317 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00ce60 con 0x7f34ec104230 2024-04-17T07:54:45.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.864+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1a2910 con 0x7f34ec104230 2024-04-17T07:54:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.865+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ec069260 con 0x7f34ec104230 2024-04-17T07:54:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.867+0000 7f34e3400640 1 -- 172.21.15.46:0/3729708317 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f34dc0037a0 con 0x7f34ec104230 2024-04-17T07:54:45.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.869+0000 7f34e3400640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34d00a3c30 0x7f34d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:45.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.869+0000 7f34e3400640 1 -- 172.21.15.46:0/3729708317 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f34dc0d02c0 con 0x7f34ec104230 2024-04-17T07:54:45.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.869+0000 7f34ea000640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34d00a3c30 0x7f34d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.870+0000 7f34ea000640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34d00a3c30 0x7f34d00a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34d802c8e0 tx=0x7f34d802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:45.873+0000 7f34e3400640 1 -- 172.21.15.46:0/3729708317 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f34dc09d0a0 con 0x7f34ec104230 2024-04-17T07:54:46.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.164+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34ec10b4f0 con 0x7f34ec104230 2024-04-17T07:54:46.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.165+0000 7f34e3400640 1 -- 172.21.15.46:0/3729708317 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f34ec10b4f0 con 0x7f34ec104230 2024-04-17T07:54:46.164 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:46.165 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-17T07:54:46.165 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-04-17T07:54:46.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34d00a3c30 msgr2=0x7f34d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:46.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34d00a3c30 0x7f34d00a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34d802c8e0 tx=0x7f34d802df90 comp rx=0 tx=0).stop 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec104230 msgr2=0x7f34ec19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec104230 0x7f34ec19a490 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00e9c0 tx=0x7f34dc00ee80 comp rx=0 tx=0).stop 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 shutdown_connections 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34d00a3c30 0x7f34d00a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34ec105de0 0x7f34ec1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34ec105420 0x7f34ec19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 --2- 172.21.15.46:0/3729708317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34ec104230 0x7f34ec19a490 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 >> 172.21.15.46:0/3729708317 conn(0x7f34ec0ff7e0 msgr2=0x7f34ec1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 shutdown_connections 2024-04-17T07:54:46.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:46.168+0000 7f34f1600640 1 -- 172.21.15.46:0/3729708317 wait complete. 2024-04-17T07:54:46.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:46.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:46.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:46.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:46.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: Deploying daemon mds.j on smithi046 2024-04-17T07:54:46.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3729708317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:46.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6832/2557309272,v1:172.21.15.53:6833/2557309272] up:boot 2024-04-17T07:54:46.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: fsmap 2 up:standby 2024-04-17T07:54:46.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:46 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T07:54:46.793 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.793 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.793 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:46.793 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: Deploying daemon mds.j on smithi046 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3729708317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6832/2557309272,v1:172.21.15.53:6833/2557309272] up:boot 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: fsmap 2 up:standby 2024-04-17T07:54:46.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:46 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: Deploying daemon mds.j on smithi046 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3729708317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6832/2557309272,v1:172.21.15.53:6833/2557309272] up:boot 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: fsmap 2 up:standby 2024-04-17T07:54:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:46 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T07:54:48.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:48.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 -- 172.21.15.46:0/1024008884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105430 msgr2=0x7f38bc1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1024008884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105430 0x7f38bc1058b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009900 tx=0x7f38a402bc40 comp rx=0 tx=0).stop 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 -- 172.21.15.46:0/1024008884 shutdown_connections 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1024008884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc105df0 0x7f38bc10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1024008884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105430 0x7f38bc1058b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1024008884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc104240 0x7f38bc104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 -- 172.21.15.46:0/1024008884 >> 172.21.15.46:0/1024008884 conn(0x7f38bc0ff7b0 msgr2=0x7f38bc101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 -- 172.21.15.46:0/1024008884 shutdown_connections 2024-04-17T07:54:48.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.602+0000 7f38c0e00640 1 -- 172.21.15.46:0/1024008884 wait complete. 2024-04-17T07:54:48.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.603+0000 7f38c0e00640 1 Processor -- start 2024-04-17T07:54:48.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.603+0000 7f38c0e00640 1 -- start start 2024-04-17T07:54:48.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38c0e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc104240 0x7f38bc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38c0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc105430 0x7f38bc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38c0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105df0 0x7f38bc1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38c0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc10fe60 con 0x7f38bc105430 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38c0e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc10fce0 con 0x7f38bc104240 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38c0e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc10ffe0 con 0x7f38bc105df0 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc104240 0x7f38bc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc105430 0x7f38bc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38b9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc104240 0x7f38bc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46054/0 (socket says 172.21.15.46:46054) 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38b9e00640 1 -- 172.21.15.46:0/1938162449 learned_addr learned my addr 172.21.15.46:0/1938162449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:48.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.604+0000 7f38ba800640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105df0 0x7f38bc1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b9e00640 1 -- 172.21.15.46:0/1938162449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105df0 msgr2=0x7f38bc1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b9e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105df0 0x7f38bc1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b9e00640 1 -- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc105430 msgr2=0x7f38bc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b9e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc105430 0x7f38bc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b9e00640 1 -- 172.21.15.46:0/1938162449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc1a20a0 con 0x7f38bc104240 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b9e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc104240 0x7f38bc19a480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00e9c0 tx=0x7f38ac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b2a00640 1 -- 172.21.15.46:0/1938162449 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00cd80 con 0x7f38bc104240 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b2a00640 1 -- 172.21.15.46:0/1938162449 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38ac0035f0 con 0x7f38bc104240 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc1a2390 con 0x7f38bc104240 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.605+0000 7f38b2a00640 1 -- 172.21.15.46:0/1938162449 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac010690 con 0x7f38bc104240 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.606+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc1a28b0 con 0x7f38bc104240 2024-04-17T07:54:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.607+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc10b5d0 con 0x7f38bc104240 2024-04-17T07:54:48.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.609+0000 7f38b2a00640 1 -- 172.21.15.46:0/1938162449 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f38ac0037a0 con 0x7f38bc104240 2024-04-17T07:54:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.610+0000 7f38b2a00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38a00a3bb0 0x7f38a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.610+0000 7f38b2a00640 1 -- 172.21.15.46:0/1938162449 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38ac0d0400 con 0x7f38bc104240 2024-04-17T07:54:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.610+0000 7f38b9400640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38a00a3bb0 0x7f38a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:48.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.611+0000 7f38b9400640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38a00a3bb0 0x7f38a00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f38a402c7c0 tx=0x7f38a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:48.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.614+0000 7f38b2a00640 1 -- 172.21.15.46:0/1938162449 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f38ac09d1e0 con 0x7f38bc104240 2024-04-17T07:54:48.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:48 smithi062 ceph-mon[36507]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:48.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:48 smithi046 ceph-mon[32038]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:48.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.876+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38bc02d5d0 con 0x7f38bc104240 2024-04-17T07:54:48.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.876+0000 7f38b2a00640 1 -- 172.21.15.46:0/1938162449 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f38ac09cae0 con 0x7f38bc104240 2024-04-17T07:54:48.876 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:48.876 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:54:48.876 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-04-17T07:54:48.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38a00a3bb0 msgr2=0x7f38a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:48.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38a00a3bb0 0x7f38a00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f38a402c7c0 tx=0x7f38a4005fd0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc104240 msgr2=0x7f38bc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc104240 0x7f38bc19a480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00e9c0 tx=0x7f38ac00ee80 comp rx=0 tx=0).stop 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 shutdown_connections 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38a00a3bb0 0x7f38a00a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38bc105df0 0x7f38bc1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc105430 0x7f38bc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 --2- 172.21.15.46:0/1938162449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38bc104240 0x7f38bc19a480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 >> 172.21.15.46:0/1938162449 conn(0x7f38bc0ff7b0 msgr2=0x7f38bc1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 shutdown_connections 2024-04-17T07:54:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:48.879+0000 7f38c0e00640 1 -- 172.21.15.46:0/1938162449 wait complete. 2024-04-17T07:54:48.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:48 smithi053 ceph-mon[36437]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:49.549 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 07:54:49 smithi046 systemd[1]: Started Ceph mds.j for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:54:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:49 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1938162449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:49.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:49 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:49.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:49 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1938162449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:49.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:49.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:49.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:49 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:49 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1938162449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:49.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:49 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:50.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:50 smithi062 ceph-mon[36507]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:50 smithi062 ceph-mon[36507]: Deploying daemon mds.d on smithi046 2024-04-17T07:54:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:50 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.46:6834/1217684239,v1:172.21.15.46:6835/1217684239] up:boot 2024-04-17T07:54:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:50 smithi062 ceph-mon[36507]: fsmap 3 up:standby 2024-04-17T07:54:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:50 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T07:54:50.818 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:50 smithi046 ceph-mon[32038]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:50.818 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:50 smithi046 ceph-mon[32038]: Deploying daemon mds.d on smithi046 2024-04-17T07:54:50.818 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:50 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.46:6834/1217684239,v1:172.21.15.46:6835/1217684239] up:boot 2024-04-17T07:54:50.818 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:50 smithi046 ceph-mon[32038]: fsmap 3 up:standby 2024-04-17T07:54:50.818 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:50 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T07:54:50.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:50 smithi053 ceph-mon[36437]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:50 smithi053 ceph-mon[36437]: Deploying daemon mds.d on smithi046 2024-04-17T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:50 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.46:6834/1217684239,v1:172.21.15.46:6835/1217684239] up:boot 2024-04-17T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:50 smithi053 ceph-mon[36437]: fsmap 3 up:standby 2024-04-17T07:54:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:50 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1790473117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103210 msgr2=0x7fdcd0103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1790473117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103210 0x7fdcd0103690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009900 tx=0x7fdcbc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1790473117 shutdown_connections 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1790473117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcd0103bd0 0x7fdcd010a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1790473117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103210 0x7fdcd0103690 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1790473117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd0102020 0x7fdcd0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1790473117 >> 172.21.15.46:0/1790473117 conn(0x7fdcd00fd590 msgr2=0x7fdcd00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1790473117 shutdown_connections 2024-04-17T07:54:51.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.100+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1790473117 wait complete. 2024-04-17T07:54:51.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.101+0000 7fdcd7a00640 1 Processor -- start 2024-04-17T07:54:51.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.101+0000 7fdcd7a00640 1 -- start start 2024-04-17T07:54:51.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd7a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd0102020 0x7fdcd0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:51.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd7a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcd0103210 0x7fdcd01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:51.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd7a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103bd0 0x7fdcd019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd7a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010dc90 con 0x7fdcd0103bd0 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd7a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010db10 con 0x7fdcd0102020 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd7a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010de10 con 0x7fdcd0103210 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103bd0 0x7fdcd019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdccfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcd0103210 0x7fdcd01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103bd0 0x7fdcd019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43264/0 (socket says 172.21.15.46:43264) 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd5000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd0102020 0x7fdcd0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.102+0000 7fdcd5a00640 1 -- 172.21.15.46:0/1972889989 learned_addr learned my addr 172.21.15.46:0/1972889989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccfe00640 1 -- 172.21.15.46:0/1972889989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd0102020 msgr2=0x7fdcd0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccfe00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd0102020 0x7fdcd0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccfe00640 1 -- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103bd0 msgr2=0x7fdcd019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccfe00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103bd0 0x7fdcd019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccfe00640 1 -- 172.21.15.46:0/1972889989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd019fe80 con 0x7fdcd0103210 2024-04-17T07:54:51.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccfe00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcd0103210 0x7fdcd01987a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc036040 tx=0x7fdcbc02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:51.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccd600640 1 -- 172.21.15.46:0/1972889989 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc043020 con 0x7fdcd0103210 2024-04-17T07:54:51.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccd600640 1 -- 172.21.15.46:0/1972889989 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc0054e0 con 0x7fdcd0103210 2024-04-17T07:54:51.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd01a0110 con 0x7fdcd0103210 2024-04-17T07:54:51.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.103+0000 7fdccd600640 1 -- 172.21.15.46:0/1972889989 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc003c30 con 0x7fdcd0103210 2024-04-17T07:54:51.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.104+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd01a0630 con 0x7fdcd0103210 2024-04-17T07:54:51.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.105+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd01092e0 con 0x7fdcd0103210 2024-04-17T07:54:51.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.107+0000 7fdccd600640 1 -- 172.21.15.46:0/1972889989 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdcbc046020 con 0x7fdcd0103210 2024-04-17T07:54:51.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.108+0000 7fdccd600640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdcb80a3bb0 0x7fdcb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:51.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.108+0000 7fdccd600640 1 -- 172.21.15.46:0/1972889989 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdcbc0f1fb0 con 0x7fdcd0103210 2024-04-17T07:54:51.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.108+0000 7fdcd5000640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdcb80a3bb0 0x7fdcb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:51.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.109+0000 7fdcd5000640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdcb80a3bb0 0x7fdcb80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0103070 tx=0x7fdcc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.113+0000 7fdccd600640 1 -- 172.21.15.46:0/1972889989 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdcbc04d020 con 0x7fdcd0103210 2024-04-17T07:54:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.364+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcd002d5d0 con 0x7fdcd0103210 2024-04-17T07:54:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.365+0000 7fdccd600640 1 -- 172.21.15.46:0/1972889989 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fdcbc0bed90 con 0x7fdcd0103210 2024-04-17T07:54:51.364 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:51.364 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:54:51.365 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 4 2024-04-17T07:54:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdcb80a3bb0 msgr2=0x7fdcb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdcb80a3bb0 0x7fdcb80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0103070 tx=0x7fdcc4005f90 comp rx=0 tx=0).stop 2024-04-17T07:54:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcd0103210 msgr2=0x7fdcd01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcd0103210 0x7fdcd01987a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc036040 tx=0x7fdcbc02c650 comp rx=0 tx=0).stop 2024-04-17T07:54:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 shutdown_connections 2024-04-17T07:54:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdcb80a3bb0 0x7fdcb80a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcd0103bd0 0x7fdcd019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcd0103210 0x7fdcd01987a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 --2- 172.21.15.46:0/1972889989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd0102020 0x7fdcd0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 >> 172.21.15.46:0/1972889989 conn(0x7fdcd00fd590 msgr2=0x7fdcd01064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 shutdown_connections 2024-04-17T07:54:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:51.368+0000 7fdcd7a00640 1 -- 172.21.15.46:0/1972889989 wait complete. 2024-04-17T07:54:51.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:51 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1972889989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:51.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:51 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1972889989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:52.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:51 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1972889989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:52.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:52 smithi046 ceph-mon[32038]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:52 smithi053 ceph-mon[36437]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:52 smithi062 ceph-mon[36507]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:53.197 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 07:54:52 smithi046 podman[72816]: 2024-04-17 07:54:52.899112133 +0000 UTC m=+0.145081713 container create af92cf60c6f9e04b3cd6a579d402bba13c0dc798a403ff3f8916bd92659af4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:54:53.197 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 07:54:53 smithi046 podman[72816]: 2024-04-17 07:54:53.196648838 +0000 UTC m=+0.442618461 container init af92cf60c6f9e04b3cd6a579d402bba13c0dc798a403ff3f8916bd92659af4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-17T07:54:53.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:53.479 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 07:54:53 smithi046 podman[72816]: 2024-04-17 07:54:53.207919193 +0000 UTC m=+0.453888780 container start af92cf60c6f9e04b3cd6a579d402bba13c0dc798a403ff3f8916bd92659af4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-17T07:54:53.479 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 07:54:53 smithi046 ceph-mds[72835]: -- 172.21.15.46:0/1600135327 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ba9c43a40 con 0x564ba9c16800 2024-04-17T07:54:53.480 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 07:54:53 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d[72831]: starting mds.d at 2024-04-17T07:54:53.480 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 07:54:53 smithi046 bash[72816]: af92cf60c6f9e04b3cd6a579d402bba13c0dc798a403ff3f8916bd92659af4d1 2024-04-17T07:54:53.480 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 07:54:53 smithi046 systemd[1]: Started Ceph mds.d for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:54:53.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 -- 172.21.15.46:0/204040239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc720105c70 msgr2=0x7fc72010c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 --2- 172.21.15.46:0/204040239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc720105c70 0x7fc72010c500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc718009ca0 tx=0x7fc71802c100 comp rx=0 tx=0).stop 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 -- 172.21.15.46:0/204040239 shutdown_connections 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 --2- 172.21.15.46:0/204040239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc720105c70 0x7fc72010c500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 --2- 172.21.15.46:0/204040239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7201052b0 0x7fc720105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 --2- 172.21.15.46:0/204040239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7201040c0 0x7fc7201044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 -- 172.21.15.46:0/204040239 >> 172.21.15.46:0/204040239 conn(0x7fc7200ff630 msgr2=0x7fc720101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 -- 172.21.15.46:0/204040239 shutdown_connections 2024-04-17T07:54:53.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.635+0000 7fc71fe00640 1 -- 172.21.15.46:0/204040239 wait complete. 2024-04-17T07:54:53.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.636+0000 7fc71fe00640 1 Processor -- start 2024-04-17T07:54:53.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.636+0000 7fc71fe00640 1 -- start start 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.636+0000 7fc71fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7201040c0 0x7fc72019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.636+0000 7fc71fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7201052b0 0x7fc72019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.636+0000 7fc71fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc720105c70 0x7fc7201a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc72010fd50 con 0x7fc720105c70 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc72010fbd0 con 0x7fc7201052b0 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc72010fed0 con 0x7fc7201040c0 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7201040c0 0x7fc72019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7201040c0 0x7fc72019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:57908/0 (socket says 172.21.15.46:57908) 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 -- 172.21.15.46:0/4191613366 learned_addr learned my addr 172.21.15.46:0/4191613366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71e000640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7201052b0 0x7fc72019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71f400640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc720105c70 0x7fc7201a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 -- 172.21.15.46:0/4191613366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7201052b0 msgr2=0x7fc72019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7201052b0 0x7fc72019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 -- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc720105c70 msgr2=0x7fc7201a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc720105c70 0x7fc7201a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71ea00640 1 -- 172.21.15.46:0/4191613366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7201a1f40 con 0x7fc7201040c0 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.637+0000 7fc71e000640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7201052b0 0x7fc72019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.638+0000 7fc71ea00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7201040c0 0x7fc72019a320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc71000e9c0 tx=0x7fc71000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.638+0000 7fc717400640 1 -- 172.21.15.46:0/4191613366 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71000cd80 con 0x7fc7201040c0 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.638+0000 7fc717400640 1 -- 172.21.15.46:0/4191613366 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7100035f0 con 0x7fc7201040c0 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.638+0000 7fc717400640 1 -- 172.21.15.46:0/4191613366 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710010690 con 0x7fc7201040c0 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.638+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7201a2230 con 0x7fc7201040c0 2024-04-17T07:54:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.638+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7201a2730 con 0x7fc7201040c0 2024-04-17T07:54:53.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.639+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72010b380 con 0x7fc7201040c0 2024-04-17T07:54:53.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.641+0000 7fc717400640 1 -- 172.21.15.46:0/4191613366 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc7100037a0 con 0x7fc7201040c0 2024-04-17T07:54:53.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.642+0000 7fc717400640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7040a3c20 0x7fc7040a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:53.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.642+0000 7fc717400640 1 -- 172.21.15.46:0/4191613366 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc7100d0360 con 0x7fc7201040c0 2024-04-17T07:54:53.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.643+0000 7fc71e000640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7040a3c20 0x7fc7040a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:53.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.643+0000 7fc71e000640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7040a3c20 0x7fc7040a60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc72019b830 tx=0x7fc70c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:53.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.647+0000 7fc717400640 1 -- 172.21.15.46:0/4191613366 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc71009d140 con 0x7fc7201040c0 2024-04-17T07:54:53.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.894+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc720062b60 con 0x7fc7201040c0 2024-04-17T07:54:53.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.895+0000 7fc717400640 1 -- 172.21.15.46:0/4191613366 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fc71009ca40 con 0x7fc7201040c0 2024-04-17T07:54:53.895 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:53.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:54:53.895 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 5 2024-04-17T07:54:53.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.898+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7040a3c20 msgr2=0x7fc7040a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:53.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.898+0000 7fc71fe00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7040a3c20 0x7fc7040a60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc72019b830 tx=0x7fc70c005f90 comp rx=0 tx=0).stop 2024-04-17T07:54:53.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.898+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7201040c0 msgr2=0x7fc72019a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:53.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.898+0000 7fc71fe00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7201040c0 0x7fc72019a320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc71000e9c0 tx=0x7fc71000ee80 comp rx=0 tx=0).stop 2024-04-17T07:54:53.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 shutdown_connections 2024-04-17T07:54:53.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7040a3c20 0x7fc7040a60e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc720105c70 0x7fc7201a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7201052b0 0x7fc72019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 --2- 172.21.15.46:0/4191613366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7201040c0 0x7fc72019a320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:53.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 >> 172.21.15.46:0/4191613366 conn(0x7fc7200ff630 msgr2=0x7fc720108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:53.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 shutdown_connections 2024-04-17T07:54:53.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:53.899+0000 7fc71fe00640 1 -- 172.21.15.46:0/4191613366 wait complete. 2024-04-17T07:54:54.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: Deploying daemon mds.e on smithi053 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.46:6836/1258153884,v1:172.21.15.46:6837/1258153884] up:boot 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: fsmap 4 up:standby 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T07:54:54.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:54 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4191613366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: Deploying daemon mds.e on smithi053 2024-04-17T07:54:54.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:54.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.46:6836/1258153884,v1:172.21.15.46:6837/1258153884] up:boot 2024-04-17T07:54:54.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: fsmap 4 up:standby 2024-04-17T07:54:54.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T07:54:54.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:54 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4191613366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: Deploying daemon mds.e on smithi053 2024-04-17T07:54:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.46:6836/1258153884,v1:172.21.15.46:6837/1258153884] up:boot 2024-04-17T07:54:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: fsmap 4 up:standby 2024-04-17T07:54:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T07:54:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:54 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4191613366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:55.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 -- 172.21.15.46:0/460988789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103a00 msgr2=0x7f527010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 --2- 172.21.15.46:0/460988789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103a00 0x7f527010a320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5268009ca0 tx=0x7f526802c100 comp rx=0 tx=0).stop 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 -- 172.21.15.46:0/460988789 shutdown_connections 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 --2- 172.21.15.46:0/460988789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103a00 0x7f527010a320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 --2- 172.21.15.46:0/460988789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5270103040 0x7f52701034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 --2- 172.21.15.46:0/460988789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 0x7f5270102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 -- 172.21.15.46:0/460988789 >> 172.21.15.46:0/460988789 conn(0x7f52700fd420 msgr2=0x7f52700ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 -- 172.21.15.46:0/460988789 shutdown_connections 2024-04-17T07:54:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.127+0000 7f5275000640 1 -- 172.21.15.46:0/460988789 wait complete. 2024-04-17T07:54:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 Processor -- start 2024-04-17T07:54:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 -- start start 2024-04-17T07:54:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 0x7f5270198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103040 0x7f5270198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5270103a00 0x7f527019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f527010db40 con 0x7f5270103040 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f527010d9c0 con 0x7f5270101e60 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.129+0000 7f5275000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f527010dcc0 con 0x7f5270103a00 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 0x7f5270198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 0x7f5270198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46142/0 (socket says 172.21.15.46:46142) 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 -- 172.21.15.46:0/4047126152 learned_addr learned my addr 172.21.15.46:0/4047126152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526ea00640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103040 0x7f5270198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526fe00640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5270103a00 0x7f527019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 -- 172.21.15.46:0/4047126152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5270103a00 msgr2=0x7f527019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5270103a00 0x7f527019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 -- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103040 msgr2=0x7f5270198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103040 0x7f5270198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 -- 172.21.15.46:0/4047126152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527019fd40 con 0x7f5270101e60 2024-04-17T07:54:56.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526ea00640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103040 0x7f5270198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:54:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.130+0000 7f526f400640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 0x7f5270198120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f526000e9c0 tx=0x7f526000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.131+0000 7f5267e00640 1 -- 172.21.15.46:0/4047126152 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526000cde0 con 0x7f5270101e60 2024-04-17T07:54:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.131+0000 7f5267e00640 1 -- 172.21.15.46:0/4047126152 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5260010910 con 0x7f5270101e60 2024-04-17T07:54:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.131+0000 7f5267e00640 1 -- 172.21.15.46:0/4047126152 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260018a00 con 0x7f5270101e60 2024-04-17T07:54:56.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.131+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527019ffd0 con 0x7f5270101e60 2024-04-17T07:54:56.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.131+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52701a04e0 con 0x7f5270101e60 2024-04-17T07:54:56.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.132+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5270109130 con 0x7f5270101e60 2024-04-17T07:54:56.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.134+0000 7f5267e00640 1 -- 172.21.15.46:0/4047126152 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f526001f020 con 0x7f5270101e60 2024-04-17T07:54:56.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.135+0000 7f5267e00640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52540a3c20 0x7f52540a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:56.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.135+0000 7f5267e00640 1 -- 172.21.15.46:0/4047126152 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f52600d01e0 con 0x7f5270101e60 2024-04-17T07:54:56.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.136+0000 7f526ea00640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52540a3c20 0x7f52540a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:56.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.136+0000 7f526ea00640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52540a3c20 0x7f52540a60e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5270199630 tx=0x7f525c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:56.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.140+0000 7f5267e00640 1 -- 172.21.15.46:0/4047126152 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f526009cfc0 con 0x7f5270101e60 2024-04-17T07:54:56.221 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 07:54:56 smithi053 systemd[1]: Started Ceph mds.e for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:54:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.394+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f527002d5d0 con 0x7f5270101e60 2024-04-17T07:54:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.395+0000 7f5267e00640 1 -- 172.21.15.46:0/4047126152 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f527002d5d0 con 0x7f5270101e60 2024-04-17T07:54:56.394 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:56.394 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:54:56.394 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 5 2024-04-17T07:54:56.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52540a3c20 msgr2=0x7f52540a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52540a3c20 0x7f52540a60e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5270199630 tx=0x7f525c005f90 comp rx=0 tx=0).stop 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 msgr2=0x7f5270198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 0x7f5270198120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f526000e9c0 tx=0x7f526000ee80 comp rx=0 tx=0).stop 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 shutdown_connections 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52540a3c20 0x7f52540a60e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5270103a00 0x7f527019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5270103040 0x7f5270198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 --2- 172.21.15.46:0/4047126152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5270101e60 0x7f5270198120 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 >> 172.21.15.46:0/4047126152 conn(0x7f52700fd420 msgr2=0x7f5270106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 shutdown_connections 2024-04-17T07:54:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:56.397+0000 7f5275000640 1 -- 172.21.15.46:0/4047126152 wait complete. 2024-04-17T07:54:57.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:57.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4047126152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6834/3940284604,v1:172.21.15.53:6835/3940284604] up:boot 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: fsmap 5 up:standby 2024-04-17T07:54:57.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:57 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4047126152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6834/3940284604,v1:172.21.15.53:6835/3940284604] up:boot 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: fsmap 5 up:standby 2024-04-17T07:54:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:57 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T07:54:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:54:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:54:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:54:57.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:54:57.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4047126152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:57.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6834/3940284604,v1:172.21.15.53:6835/3940284604] up:boot 2024-04-17T07:54:57.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: fsmap 5 up:standby 2024-04-17T07:54:57.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:57 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T07:54:58.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:54:58.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:58 smithi053 ceph-mon[36437]: Deploying daemon mds.f on smithi062 2024-04-17T07:54:58.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:58 smithi046 ceph-mon[32038]: Deploying daemon mds.f on smithi062 2024-04-17T07:54:58.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:58 smithi062 ceph-mon[36507]: Deploying daemon mds.f on smithi062 2024-04-17T07:54:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 -- 172.21.15.46:0/2027643418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901040c0 msgr2=0x7fed901044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 --2- 172.21.15.46:0/2027643418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901040c0 0x7fed901044e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fed80009900 tx=0x7fed8002bc40 comp rx=0 tx=0).stop 2024-04-17T07:54:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 -- 172.21.15.46:0/2027643418 shutdown_connections 2024-04-17T07:54:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 --2- 172.21.15.46:0/2027643418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed90105c70 0x7fed9010c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 --2- 172.21.15.46:0/2027643418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed901052b0 0x7fed90105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 --2- 172.21.15.46:0/2027643418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901040c0 0x7fed901044e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 -- 172.21.15.46:0/2027643418 >> 172.21.15.46:0/2027643418 conn(0x7fed900ff630 msgr2=0x7fed90101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 -- 172.21.15.46:0/2027643418 shutdown_connections 2024-04-17T07:54:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.646+0000 7fed8f200640 1 -- 172.21.15.46:0/2027643418 wait complete. 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 Processor -- start 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 -- start start 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed901040c0 0x7fed9019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901052b0 0x7fed9019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed90105c70 0x7fed901a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010fda0 con 0x7fed901052b0 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010fc20 con 0x7fed901040c0 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.647+0000 7fed8f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed9010ff20 con 0x7fed90105c70 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901052b0 0x7fed9019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed90105c70 0x7fed901a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901052b0 0x7fed9019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40392/0 (socket says 172.21.15.46:40392) 2024-04-17T07:54:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8d400640 1 -- 172.21.15.46:0/3432699388 learned_addr learned my addr 172.21.15.46:0/3432699388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8de00640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed901040c0 0x7fed9019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8e800640 1 -- 172.21.15.46:0/3432699388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed901040c0 msgr2=0x7fed9019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8e800640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed901040c0 0x7fed9019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8e800640 1 -- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901052b0 msgr2=0x7fed9019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8e800640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901052b0 0x7fed9019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8e800640 1 -- 172.21.15.46:0/3432699388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed901a1fd0 con 0x7fed90105c70 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.648+0000 7fed8d400640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901052b0 0x7fed9019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:54:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.649+0000 7fed8e800640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed90105c70 0x7fed901a1960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed8800da10 tx=0x7fed8800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.649+0000 7fed86a00640 1 -- 172.21.15.46:0/3432699388 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8800bd70 con 0x7fed90105c70 2024-04-17T07:54:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.649+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901a22c0 con 0x7fed90105c70 2024-04-17T07:54:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.649+0000 7fed86a00640 1 -- 172.21.15.46:0/3432699388 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed88003e50 con 0x7fed90105c70 2024-04-17T07:54:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.649+0000 7fed86a00640 1 -- 172.21.15.46:0/3432699388 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880106a0 con 0x7fed90105c70 2024-04-17T07:54:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.649+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed901a27c0 con 0x7fed90105c70 2024-04-17T07:54:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.650+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9010b3e0 con 0x7fed90105c70 2024-04-17T07:54:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.652+0000 7fed86a00640 1 -- 172.21.15.46:0/3432699388 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fed88004b40 con 0x7fed90105c70 2024-04-17T07:54:58.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.653+0000 7fed86a00640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fed740a3ba0 0x7fed740a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:54:58.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.654+0000 7fed86a00640 1 -- 172.21.15.46:0/3432699388 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fed880d04b0 con 0x7fed90105c70 2024-04-17T07:54:58.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.654+0000 7fed8de00640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fed740a3ba0 0x7fed740a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:54:58.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.655+0000 7fed8de00640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fed740a3ba0 0x7fed740a6060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed8002c180 tx=0x7fed8002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:54:58.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.656+0000 7fed86a00640 1 -- 172.21.15.46:0/3432699388 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fed8809d290 con 0x7fed90105c70 2024-04-17T07:54:58.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.902+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed9002d5d0 con 0x7fed90105c70 2024-04-17T07:54:58.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.903+0000 7fed86a00640 1 -- 172.21.15.46:0/3432699388 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fed8809cb90 con 0x7fed90105c70 2024-04-17T07:54:58.903 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:54:58.903 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T07:54:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2024-04-17T07:54:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fed740a3ba0 msgr2=0x7fed740a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fed740a3ba0 0x7fed740a6060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed8002c180 tx=0x7fed8002cf90 comp rx=0 tx=0).stop 2024-04-17T07:54:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed90105c70 msgr2=0x7fed901a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed90105c70 0x7fed901a1960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed8800da10 tx=0x7fed8800ded0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 shutdown_connections 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fed740a3ba0 0x7fed740a6060 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed90105c70 0x7fed901a1960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed901052b0 0x7fed9019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 --2- 172.21.15.46:0/3432699388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed901040c0 0x7fed9019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 >> 172.21.15.46:0/3432699388 conn(0x7fed900ff630 msgr2=0x7fed901085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 shutdown_connections 2024-04-17T07:54:58.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:54:58.907+0000 7fed8f200640 1 -- 172.21.15.46:0/3432699388 wait complete. 2024-04-17T07:54:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:59 smithi053 ceph-mon[36437]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:54:59 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3432699388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:59.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:59 smithi062 ceph-mon[36507]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:54:59 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3432699388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:54:59.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:59 smithi046 ceph-mon[32038]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:54:59.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:54:59 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3432699388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:00.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.62:6834/2191814520,v1:172.21.15.62:6835/2191814520] up:boot 2024-04-17T07:55:00.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: fsmap 6 up:standby 2024-04-17T07:55:00.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T07:55:00.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:00.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:00.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:00 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:00.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.62:6834/2191814520,v1:172.21.15.62:6835/2191814520] up:boot 2024-04-17T07:55:00.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: fsmap 6 up:standby 2024-04-17T07:55:00.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T07:55:00.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:00.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:00.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:00 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:00.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.62:6834/2191814520,v1:172.21.15.62:6835/2191814520] up:boot 2024-04-17T07:55:00.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: fsmap 6 up:standby 2024-04-17T07:55:00.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T07:55:00.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:00.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:00.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:00.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:00 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:00.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 -- 172.21.15.46:0/1163833253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff560102fe0 msgr2=0x7ff560103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 --2- 172.21.15.46:0/1163833253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff560102fe0 0x7ff560103460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff548009900 tx=0x7ff54802bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 -- 172.21.15.46:0/1163833253 shutdown_connections 2024-04-17T07:55:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 --2- 172.21.15.46:0/1163833253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5601039a0 0x7ff56010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 --2- 172.21.15.46:0/1163833253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff560102fe0 0x7ff560103460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 --2- 172.21.15.46:0/1163833253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff560101e90 0x7ff5601022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 -- 172.21.15.46:0/1163833253 >> 172.21.15.46:0/1163833253 conn(0x7ff5600fd420 msgr2=0x7ff5600ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 -- 172.21.15.46:0/1163833253 shutdown_connections 2024-04-17T07:55:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.167+0000 7ff565200640 1 -- 172.21.15.46:0/1163833253 wait complete. 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.168+0000 7ff565200640 1 Processor -- start 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff565200640 1 -- start start 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff565200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff560101e90 0x7ff56019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff565200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff560102fe0 0x7ff56019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff565200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601039a0 0x7ff5601a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff565200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010db70 con 0x7ff5601039a0 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff565200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010d9f0 con 0x7ff560101e90 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff565200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010dcf0 con 0x7ff560102fe0 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601039a0 0x7ff5601a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff55f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff560101e90 0x7ff56019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff55f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff560101e90 0x7ff56019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60498/0 (socket says 172.21.15.46:60498) 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff55fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601039a0 0x7ff5601a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40402/0 (socket says 172.21.15.46:40402) 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff55f400640 1 -- 172.21.15.46:0/1820083994 learned_addr learned my addr 172.21.15.46:0/1820083994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.169+0000 7ff556a00640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff560102fe0 0x7ff56019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55f400640 1 -- 172.21.15.46:0/1820083994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff560102fe0 msgr2=0x7ff56019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55f400640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff560102fe0 0x7ff56019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55f400640 1 -- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601039a0 msgr2=0x7ff5601a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55f400640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601039a0 0x7ff5601a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55f400640 1 -- 172.21.15.46:0/1820083994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601a1fd0 con 0x7ff560101e90 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55fe00640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601039a0 0x7ff5601a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55f400640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff560101e90 0x7ff56019a3b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff55000ebc0 tx=0x7ff55000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.170+0000 7ff55cc00640 1 -- 172.21.15.46:0/1820083994 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550010630 con 0x7ff560101e90 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.171+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5601a22c0 con 0x7ff560101e90 2024-04-17T07:55:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.171+0000 7ff55cc00640 1 -- 172.21.15.46:0/1820083994 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff550010cb0 con 0x7ff560101e90 2024-04-17T07:55:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.171+0000 7ff55cc00640 1 -- 172.21.15.46:0/1820083994 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550018bd0 con 0x7ff560101e90 2024-04-17T07:55:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.171+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601a27e0 con 0x7ff560101e90 2024-04-17T07:55:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.172+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5601a2a40 con 0x7ff560101e90 2024-04-17T07:55:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.174+0000 7ff55cc00640 1 -- 172.21.15.46:0/1820083994 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff5500107d0 con 0x7ff560101e90 2024-04-17T07:55:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.175+0000 7ff55cc00640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff5440a3ba0 0x7ff5440a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.175+0000 7ff55cc00640 1 -- 172.21.15.46:0/1820083994 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5500d05e0 con 0x7ff560101e90 2024-04-17T07:55:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.176+0000 7ff556a00640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff5440a3ba0 0x7ff5440a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.176+0000 7ff556a00640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff5440a3ba0 0x7ff5440a6060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff54802c170 tx=0x7ff54802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.180+0000 7ff55cc00640 1 -- 172.21.15.46:0/1820083994 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff55009cd70 con 0x7ff560101e90 2024-04-17T07:55:01.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:01 smithi046 ceph-mon[32038]: Deploying daemon mds.c on smithi062 2024-04-17T07:55:01.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:01 smithi046 ceph-mon[32038]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:01.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.431+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5601a2c10 con 0x7ff560101e90 2024-04-17T07:55:01.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.432+0000 7ff55cc00640 1 -- 172.21.15.46:0/1820083994 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7ff5500a1070 con 0x7ff560101e90 2024-04-17T07:55:01.432 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:01.432 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-17T07:55:01.432 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-04-17T07:55:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.434+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff5440a3ba0 msgr2=0x7ff5440a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.434+0000 7ff565200640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff5440a3ba0 0x7ff5440a6060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff54802c170 tx=0x7ff54802df90 comp rx=0 tx=0).stop 2024-04-17T07:55:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff560101e90 msgr2=0x7ff56019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff560101e90 0x7ff56019a3b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff55000ebc0 tx=0x7ff55000c6a0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 shutdown_connections 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff5440a3ba0 0x7ff5440a6060 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601039a0 0x7ff5601a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff560102fe0 0x7ff56019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 --2- 172.21.15.46:0/1820083994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff560101e90 0x7ff56019a3b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 >> 172.21.15.46:0/1820083994 conn(0x7ff5600fd420 msgr2=0x7ff560074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 shutdown_connections 2024-04-17T07:55:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:01.435+0000 7ff565200640 1 -- 172.21.15.46:0/1820083994 wait complete. 2024-04-17T07:55:01.459 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:01 smithi062 ceph-mon[36507]: Deploying daemon mds.c on smithi062 2024-04-17T07:55:01.459 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:01 smithi062 ceph-mon[36507]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:01.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:01 smithi053 ceph-mon[36437]: Deploying daemon mds.c on smithi062 2024-04-17T07:55:01.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:01 smithi053 ceph-mon[36437]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:01.803 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:01 smithi062 systemd[1]: Starting Ceph mds.c for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:55:02.171 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:01 smithi062 podman[67666]: 2024-04-17 07:55:01.774014885 +0000 UTC m=+0.018281533 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:55:02.172 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:01 smithi062 podman[67666]: 2024-04-17 07:55:01.905670927 +0000 UTC m=+0.149937575 container create 0d86076c3e17edd1129697053a9be14a3d80bc7ec93f36a2686c65a5091d2360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-17T07:55:02.172 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:02 smithi062 podman[67666]: 2024-04-17 07:55:02.173007514 +0000 UTC m=+0.417274163 container init 0d86076c3e17edd1129697053a9be14a3d80bc7ec93f36a2686c65a5091d2360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-17T07:55:02.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:02 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1820083994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:02.553 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:02 smithi062 podman[67666]: 2024-04-17 07:55:02.175827668 +0000 UTC m=+0.420094314 container start 0d86076c3e17edd1129697053a9be14a3d80bc7ec93f36a2686c65a5091d2360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:55:02.553 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:02 smithi062 ceph-mds[67686]: -- 172.21.15.62:0/1888383279 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561739639a40 con 0x56173960c800 2024-04-17T07:55:02.553 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c[67682]: starting mds.c at 2024-04-17T07:55:02.553 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:02 smithi062 bash[67666]: 0d86076c3e17edd1129697053a9be14a3d80bc7ec93f36a2686c65a5091d2360 2024-04-17T07:55:02.553 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 07:55:02 smithi062 systemd[1]: Started Ceph mds.c for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:55:02.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:02 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1820083994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:02.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:02 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1820083994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.62:6836/956269488,v1:172.21.15.62:6837/956269488] up:boot 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: fsmap 7 up:standby 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:03.359 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:03.360 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:03 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:03.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:03.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.62:6836/956269488,v1:172.21.15.62:6837/956269488] up:boot 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: fsmap 7 up:standby 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:03 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.62:6836/956269488,v1:172.21.15.62:6837/956269488] up:boot 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: fsmap 7 up:standby 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:03.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:03 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 -- 172.21.15.46:0/2674272519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181040c0 msgr2=0x7fee181044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 --2- 172.21.15.46:0/2674272519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181040c0 0x7fee181044e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009900 tx=0x7fee0c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 -- 172.21.15.46:0/2674272519 shutdown_connections 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 --2- 172.21.15.46:0/2674272519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee18105c70 0x7fee1810c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 --2- 172.21.15.46:0/2674272519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181052b0 0x7fee18105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 --2- 172.21.15.46:0/2674272519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181040c0 0x7fee181044e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 -- 172.21.15.46:0/2674272519 >> 172.21.15.46:0/2674272519 conn(0x7fee180ff630 msgr2=0x7fee18101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 -- 172.21.15.46:0/2674272519 shutdown_connections 2024-04-17T07:55:03.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.690+0000 7fee1f200640 1 -- 172.21.15.46:0/2674272519 wait complete. 2024-04-17T07:55:03.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.691+0000 7fee1f200640 1 Processor -- start 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.691+0000 7fee1f200640 1 -- start start 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.691+0000 7fee1f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181040c0 0x7fee1819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181052b0 0x7fee1819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee18105c70 0x7fee181a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee1810fd50 con 0x7fee181052b0 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee1810fbd0 con 0x7fee18105c70 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee1810fed0 con 0x7fee181040c0 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181040c0 0x7fee1819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181052b0 0x7fee1819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:03.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181040c0 0x7fee1819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37234/0 (socket says 172.21.15.46:37234) 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 -- 172.21.15.46:0/47756359 learned_addr learned my addr 172.21.15.46:0/47756359 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1e800640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee18105c70 0x7fee181a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 -- 172.21.15.46:0/47756359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee18105c70 msgr2=0x7fee181a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee18105c70 0x7fee181a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 -- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181052b0 msgr2=0x7fee1819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181052b0 0x7fee1819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1de00640 1 -- 172.21.15.46:0/47756359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee181a1fa0 con 0x7fee181040c0 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1e800640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee18105c70 0x7fee181a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.692+0000 7fee1d400640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181052b0 0x7fee1819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.693+0000 7fee1de00640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181040c0 0x7fee1819a380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee0c000c00 tx=0x7fee0c02c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.693+0000 7fee12a00640 1 -- 172.21.15.46:0/47756359 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c041070 con 0x7fee181040c0 2024-04-17T07:55:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.693+0000 7fee12a00640 1 -- 172.21.15.46:0/47756359 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee0c004450 con 0x7fee181040c0 2024-04-17T07:55:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.693+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee181a2230 con 0x7fee181040c0 2024-04-17T07:55:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.693+0000 7fee12a00640 1 -- 172.21.15.46:0/47756359 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c034440 con 0x7fee181040c0 2024-04-17T07:55:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.693+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee181a2720 con 0x7fee181040c0 2024-04-17T07:55:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.694+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1810b3e0 con 0x7fee181040c0 2024-04-17T07:55:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.696+0000 7fee12a00640 1 -- 172.21.15.46:0/47756359 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fee0c044070 con 0x7fee181040c0 2024-04-17T07:55:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.698+0000 7fee12a00640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fee000a3ba0 0x7fee000a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.698+0000 7fee12a00640 1 -- 172.21.15.46:0/47756359 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fee0c0f1db0 con 0x7fee181040c0 2024-04-17T07:55:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.698+0000 7fee1d400640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fee000a3ba0 0x7fee000a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:03.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.698+0000 7fee1d400640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fee000a3ba0 0x7fee000a6060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee1819b890 tx=0x7fee08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.700+0000 7fee12a00640 1 -- 172.21.15.46:0/47756359 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fee0c0bec20 con 0x7fee181040c0 2024-04-17T07:55:03.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.951+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee1802d5d0 con 0x7fee181040c0 2024-04-17T07:55:03.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.952+0000 7fee12a00640 1 -- 172.21.15.46:0/47756359 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7fee0c0be520 con 0x7fee181040c0 2024-04-17T07:55:03.951 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:03.951 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-17T07:55:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-04-17T07:55:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fee000a3ba0 msgr2=0x7fee000a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fee000a3ba0 0x7fee000a6060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee1819b890 tx=0x7fee08005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181040c0 msgr2=0x7fee1819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181040c0 0x7fee1819a380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee0c000c00 tx=0x7fee0c02c640 comp rx=0 tx=0).stop 2024-04-17T07:55:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 shutdown_connections 2024-04-17T07:55:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fee000a3ba0 0x7fee000a6060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee18105c70 0x7fee181a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee181052b0 0x7fee1819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 --2- 172.21.15.46:0/47756359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee181040c0 0x7fee1819a380 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 >> 172.21.15.46:0/47756359 conn(0x7fee180ff630 msgr2=0x7fee181085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 shutdown_connections 2024-04-17T07:55:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:03.955+0000 7fee1f200640 1 -- 172.21.15.46:0/47756359 wait complete. 2024-04-17T07:55:04.466 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:04 smithi053 ceph-mon[36437]: Deploying daemon mds.h on smithi053 2024-04-17T07:55:04.466 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:04 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/47756359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:04 smithi062 ceph-mon[36507]: Deploying daemon mds.h on smithi053 2024-04-17T07:55:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:04 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/47756359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:04.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:04 smithi046 ceph-mon[32038]: Deploying daemon mds.h on smithi053 2024-04-17T07:55:04.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:04 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/47756359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:04.721 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 07:55:04 smithi053 podman[67058]: 2024-04-17 07:55:04.691040145 +0000 UTC m=+0.122397696 container create cc51d3d83da7b612b9c11f1280f5b4b9eb2181bf3b1980bb36413b2571a329b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-17T07:55:05.221 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 07:55:04 smithi053 podman[67058]: 2024-04-17 07:55:04.952554398 +0000 UTC m=+0.383911958 container init cc51d3d83da7b612b9c11f1280f5b4b9eb2181bf3b1980bb36413b2571a329b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:55:05.221 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 07:55:04 smithi053 podman[67058]: 2024-04-17 07:55:04.955280133 +0000 UTC m=+0.386637685 container start cc51d3d83da7b612b9c11f1280f5b4b9eb2181bf3b1980bb36413b2571a329b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5) 2024-04-17T07:55:05.221 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 07:55:04 smithi053 ceph-mds[67077]: -- 172.21.15.53:0/1180886017 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2f6e99a40 con 0x55d2f6e6d000 2024-04-17T07:55:05.222 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 07:55:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h[67073]: starting mds.h at 2024-04-17T07:55:05.222 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 07:55:05 smithi053 bash[67058]: cc51d3d83da7b612b9c11f1280f5b4b9eb2181bf3b1980bb36413b2571a329b6 2024-04-17T07:55:05.222 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 07:55:05 smithi053 systemd[1]: Started Ceph mds.h for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:55:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:05 smithi062 ceph-mon[36507]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:05.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:05 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:05.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:05 smithi046 ceph-mon[32038]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:05.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:05.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:05.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:05 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:05.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:05 smithi053 ceph-mon[36437]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:05.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:05.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:05.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:05.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:05 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:05.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:06.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.198+0000 7f7144800640 1 -- 172.21.15.46:0/3670941842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 msgr2=0x7f713c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:06.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.198+0000 7f7144800640 1 --2- 172.21.15.46:0/3670941842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c105840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7128009900 tx=0x7f712802bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:06.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.199+0000 7f7144800640 1 -- 172.21.15.46:0/3670941842 shutdown_connections 2024-04-17T07:55:06.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.199+0000 7f7144800640 1 --2- 172.21.15.46:0/3670941842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c105d80 0x7f713c10c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.199+0000 7f7144800640 1 --2- 172.21.15.46:0/3670941842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c105840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.199+0000 7f7144800640 1 --2- 172.21.15.46:0/3670941842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f713c1041d0 0x7f713c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.199+0000 7f7144800640 1 -- 172.21.15.46:0/3670941842 >> 172.21.15.46:0/3670941842 conn(0x7f713c0ff780 msgr2=0x7f713c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:06.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.199+0000 7f7144800640 1 -- 172.21.15.46:0/3670941842 shutdown_connections 2024-04-17T07:55:06.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.199+0000 7f7144800640 1 -- 172.21.15.46:0/3670941842 wait complete. 2024-04-17T07:55:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 Processor -- start 2024-04-17T07:55:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 -- start start 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f713c1041d0 0x7f713c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c105d80 0x7f713c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f713c10feb0 con 0x7f713c1041d0 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f713c10fd30 con 0x7f713c105d80 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.200+0000 7f7144800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f713c110030 con 0x7f713c1053c0 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f713c1041d0 0x7f713c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37244/0 (socket says 172.21.15.46:37244) 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 -- 172.21.15.46:0/3719464971 learned_addr learned my addr 172.21.15.46:0/3719464971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7142800640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c105d80 0x7f713c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 -- 172.21.15.46:0/3719464971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c105d80 msgr2=0x7f713c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c105d80 0x7f713c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 -- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f713c1041d0 msgr2=0x7f713c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f713c1041d0 0x7f713c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 -- 172.21.15.46:0/3719464971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c1a20c0 con 0x7f713c1053c0 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.201+0000 7f7141400640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c19a9e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712802c8f0 tx=0x7f7128005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.202+0000 7f7136a00640 1 -- 172.21.15.46:0/3719464971 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128041070 con 0x7f713c1053c0 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.202+0000 7f7136a00640 1 -- 172.21.15.46:0/3719464971 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71280044d0 con 0x7f713c1053c0 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.202+0000 7f7136a00640 1 -- 172.21.15.46:0/3719464971 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128034440 con 0x7f713c1053c0 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.202+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c1a2350 con 0x7f713c1053c0 2024-04-17T07:55:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.202+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c1a27f0 con 0x7f713c1053c0 2024-04-17T07:55:06.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.203+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713c10b4f0 con 0x7f713c1053c0 2024-04-17T07:55:06.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.205+0000 7f7136a00640 1 -- 172.21.15.46:0/3719464971 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7128044070 con 0x7f713c1053c0 2024-04-17T07:55:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.206+0000 7f7136a00640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f71240a3c30 0x7f71240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.206+0000 7f7136a00640 1 -- 172.21.15.46:0/3719464971 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f71280f1ac0 con 0x7f713c1053c0 2024-04-17T07:55:06.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.206+0000 7f7141e00640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f71240a3c30 0x7f71240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:06.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.207+0000 7f7141e00640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f71240a3c30 0x7f71240a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f713c105220 tx=0x7f713000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.211+0000 7f7136a00640 1 -- 172.21.15.46:0/3719464971 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f71280be250 con 0x7f713c1053c0 2024-04-17T07:55:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:06 smithi046 ceph-mon[32038]: Deploying daemon mds.i on smithi062 2024-04-17T07:55:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:06 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] up:boot 2024-04-17T07:55:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:06 smithi046 ceph-mon[32038]: fsmap 8 up:standby 2024-04-17T07:55:06.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:06 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T07:55:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.461+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f713c02d5d0 con 0x7f713c1053c0 2024-04-17T07:55:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.462+0000 7f7136a00640 1 -- 172.21.15.46:0/3719464971 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7165 (secure 0 0 0) 0x7f71280c2550 con 0x7f713c1053c0 2024-04-17T07:55:06.461 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:06.462 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:55:06.462 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.464+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f71240a3c30 msgr2=0x7f71240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.464+0000 7f7144800640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f71240a3c30 0x7f71240a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f713c105220 tx=0x7f713000b390 comp rx=0 tx=0).stop 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.464+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 msgr2=0x7f713c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.464+0000 7f7144800640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c19a9e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712802c8f0 tx=0x7f7128005d50 comp rx=0 tx=0).stop 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 shutdown_connections 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f71240a3c30 0x7f71240a60f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c105d80 0x7f713c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f713c1053c0 0x7f713c19a9e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 --2- 172.21.15.46:0/3719464971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f713c1041d0 0x7f713c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 >> 172.21.15.46:0/3719464971 conn(0x7f713c0ff780 msgr2=0x7f713c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 shutdown_connections 2024-04-17T07:55:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:06.465+0000 7f7144800640 1 -- 172.21.15.46:0/3719464971 wait complete. 2024-04-17T07:55:06.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:06 smithi062 ceph-mon[36507]: Deploying daemon mds.i on smithi062 2024-04-17T07:55:06.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:06 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] up:boot 2024-04-17T07:55:06.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:06 smithi062 ceph-mon[36507]: fsmap 8 up:standby 2024-04-17T07:55:06.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:06 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T07:55:06.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:06 smithi053 ceph-mon[36437]: Deploying daemon mds.i on smithi062 2024-04-17T07:55:06.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:06 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] up:boot 2024-04-17T07:55:06.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:06 smithi053 ceph-mon[36437]: fsmap 8 up:standby 2024-04-17T07:55:06.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:06 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T07:55:07.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:07 smithi062 ceph-mon[36507]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:07.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:07 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3719464971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:07.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:07 smithi046 ceph-mon[32038]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:07.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:07 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3719464971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:07.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:07 smithi053 ceph-mon[36437]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:07.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:07 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3719464971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:08.302 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 07:55:08 smithi062 systemd[1]: Started Ceph mds.i for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:55:08.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 -- 172.21.15.46:0/1140029284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4076920 msgr2=0x7f17c4076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1140029284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4076920 0x7f17c4076d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009900 tx=0x7f17b802bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 -- 172.21.15.46:0/1140029284 shutdown_connections 2024-04-17T07:55:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1140029284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c4078500 0x7f17c4078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1140029284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4077b60 0x7f17c4077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1140029284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4076920 0x7f17c4076d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 -- 172.21.15.46:0/1140029284 >> 172.21.15.46:0/1140029284 conn(0x7f17c40ff7e0 msgr2=0x7f17c4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 -- 172.21.15.46:0/1140029284 shutdown_connections 2024-04-17T07:55:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.690+0000 7f17cbc00640 1 -- 172.21.15.46:0/1140029284 wait complete. 2024-04-17T07:55:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.691+0000 7f17cbc00640 1 Processor -- start 2024-04-17T07:55:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.691+0000 7f17cbc00640 1 -- start start 2024-04-17T07:55:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17cbc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4076920 0x7f17c419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17cbc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c4077b60 0x7f17c419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17cbc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4078500 0x7f17c41a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17cbc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114290 con 0x7f17c4078500 2024-04-17T07:55:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17cbc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114110 con 0x7f17c4076920 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17cbc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114410 con 0x7f17c4077b60 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17c9200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4076920 0x7f17c419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17c9200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4076920 0x7f17c419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:60980/0 (socket says 172.21.15.46:60980) 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17c9200640 1 -- 172.21.15.46:0/1440030451 learned_addr learned my addr 172.21.15.46:0/1440030451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17c3e00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c4077b60 0x7f17c419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.692+0000 7f17c9c00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4078500 0x7f17c41a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c9200640 1 -- 172.21.15.46:0/1440030451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c4077b60 msgr2=0x7f17c419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c9200640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c4077b60 0x7f17c419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c9200640 1 -- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4078500 msgr2=0x7f17c41a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c9200640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4078500 0x7f17c41a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c9200640 1 -- 172.21.15.46:0/1440030451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c41a64d0 con 0x7f17c4076920 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c9200640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4076920 0x7f17c419e8b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000c00 tx=0x7f17b802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c1600640 1 -- 172.21.15.46:0/1440030451 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8043020 con 0x7f17c4076920 2024-04-17T07:55:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c41a6760 con 0x7f17c4076920 2024-04-17T07:55:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c1600640 1 -- 172.21.15.46:0/1440030451 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b80054e0 con 0x7f17c4076920 2024-04-17T07:55:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17c1600640 1 -- 172.21.15.46:0/1440030451 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8003c40 con 0x7f17c4076920 2024-04-17T07:55:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.693+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a6cb0 con 0x7f17c4076920 2024-04-17T07:55:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.695+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c4110d00 con 0x7f17c4076920 2024-04-17T07:55:08.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.696+0000 7f17c1600640 1 -- 172.21.15.46:0/1440030451 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f17b8046020 con 0x7f17c4076920 2024-04-17T07:55:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.698+0000 7f17c1600640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17ac0a3bb0 0x7f17ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.698+0000 7f17c1600640 1 -- 172.21.15.46:0/1440030451 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f17b80f1fd0 con 0x7f17c4076920 2024-04-17T07:55:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.698+0000 7f17c3e00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17ac0a3bb0 0x7f17ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:08.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.699+0000 7f17c3e00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17ac0a3bb0 0x7f17ac0a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f17b00095f0 tx=0x7f17b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.702+0000 7f17c1600640 1 -- 172.21.15.46:0/1440030451 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17b804d020 con 0x7f17c4076920 2024-04-17T07:55:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.954+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17c402d5d0 con 0x7f17c4076920 2024-04-17T07:55:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.955+0000 7f17c1600640 1 -- 172.21.15.46:0/1440030451 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7f17b80bedb0 con 0x7f17c4076920 2024-04-17T07:55:08.954 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:08.954 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:55:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 10 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17ac0a3bb0 msgr2=0x7f17ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17ac0a3bb0 0x7f17ac0a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f17b00095f0 tx=0x7f17b0005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4076920 msgr2=0x7f17c419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4076920 0x7f17c419e8b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000c00 tx=0x7f17b802c650 comp rx=0 tx=0).stop 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 shutdown_connections 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17ac0a3bb0 0x7f17ac0a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17c4078500 0x7f17c41a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c4077b60 0x7f17c419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 --2- 172.21.15.46:0/1440030451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17c4076920 0x7f17c419e8b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 >> 172.21.15.46:0/1440030451 conn(0x7f17c40ff7e0 msgr2=0x7f17c4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.959+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 shutdown_connections 2024-04-17T07:55:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:08.960+0000 7f17cbc00640 1 -- 172.21.15.46:0/1440030451 wait complete. 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.62:6838/665634381,v1:172.21.15.62:6839/665634381] up:boot 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: fsmap 9 up:standby 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:09.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:09.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:09 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1440030451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.62:6838/665634381,v1:172.21.15.62:6839/665634381] up:boot 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: fsmap 9 up:standby 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:09.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:09 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1440030451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:09.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:09.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.62:6838/665634381,v1:172.21.15.62:6839/665634381] up:boot 2024-04-17T07:55:09.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: fsmap 9 up:standby 2024-04-17T07:55:09.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T07:55:09.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:09.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:09.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:09.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:09.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:09 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1440030451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:10.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:10 smithi062 ceph-mon[36507]: Deploying daemon mds.g on smithi046 2024-04-17T07:55:10.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:10 smithi046 ceph-mon[32038]: Deploying daemon mds.g on smithi046 2024-04-17T07:55:10.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:10 smithi053 ceph-mon[36437]: Deploying daemon mds.g on smithi046 2024-04-17T07:55:10.987 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.430+0000 7fcb4a000640 1 -- 172.21.15.46:0/3572473866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c032340 con 0x7fcb4c104220 2024-04-17T07:55:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.430+0000 7fcb52600640 1 -- 172.21.15.46:0/3572473866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c104220 msgr2=0x7fcb4c104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.430+0000 7fcb52600640 1 --2- 172.21.15.46:0/3572473866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c104220 0x7fcb4c104640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009860 tx=0x7fcb3c02bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.431+0000 7fcb52600640 1 -- 172.21.15.46:0/3572473866 shutdown_connections 2024-04-17T07:55:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.431+0000 7fcb52600640 1 --2- 172.21.15.46:0/3572473866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c105dd0 0x7fcb4c10c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.431+0000 7fcb52600640 1 --2- 172.21.15.46:0/3572473866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb4c105410 0x7fcb4c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.431+0000 7fcb52600640 1 --2- 172.21.15.46:0/3572473866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c104220 0x7fcb4c104640 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.431+0000 7fcb52600640 1 -- 172.21.15.46:0/3572473866 >> 172.21.15.46:0/3572473866 conn(0x7fcb4c0ff7b0 msgr2=0x7fcb4c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.431+0000 7fcb52600640 1 -- 172.21.15.46:0/3572473866 shutdown_connections 2024-04-17T07:55:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.431+0000 7fcb52600640 1 -- 172.21.15.46:0/3572473866 wait complete. 2024-04-17T07:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 Processor -- start 2024-04-17T07:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 -- start start 2024-04-17T07:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104220 0x7fcb4c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c105410 0x7fcb4c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb4c105dd0 0x7fcb4c1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c076970 con 0x7fcb4c105410 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c0767f0 con 0x7fcb4c105dd0 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb52600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c076af0 con 0x7fcb4c104220 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb4aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c105410 0x7fcb4c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb4aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c105410 0x7fcb4c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34768/0 (socket says 172.21.15.46:34768) 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb4aa00640 1 -- 172.21.15.46:0/4129149700 learned_addr learned my addr 172.21.15.46:0/4129149700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb4be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb4c105dd0 0x7fcb4c1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.432+0000 7fcb4b400640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104220 0x7fcb4c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb4be00640 1 -- 172.21.15.46:0/4129149700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104220 msgr2=0x7fcb4c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb4be00640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104220 0x7fcb4c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb4be00640 1 -- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c105410 msgr2=0x7fcb4c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb4be00640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c105410 0x7fcb4c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb4be00640 1 -- 172.21.15.46:0/4129149700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c1a6530 con 0x7fcb4c105dd0 2024-04-17T07:55:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb4aa00640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c105410 0x7fcb4c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb4be00640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb4c105dd0 0x7fcb4c1a5ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400efc0 tx=0x7fcb4400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.433+0000 7fcb43e00640 1 -- 172.21.15.46:0/4129149700 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44003970 con 0x7fcb4c105dd0 2024-04-17T07:55:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.434+0000 7fcb43e00640 1 -- 172.21.15.46:0/4129149700 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb44004020 con 0x7fcb4c105dd0 2024-04-17T07:55:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.434+0000 7fcb43e00640 1 -- 172.21.15.46:0/4129149700 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400b9f0 con 0x7fcb4c105dd0 2024-04-17T07:55:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.434+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c1a6820 con 0x7fcb4c105dd0 2024-04-17T07:55:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.434+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c1a6d20 con 0x7fcb4c105dd0 2024-04-17T07:55:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.435+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c1a6f80 con 0x7fcb4c105dd0 2024-04-17T07:55:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.437+0000 7fcb43e00640 1 -- 172.21.15.46:0/4129149700 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb4401d020 con 0x7fcb4c105dd0 2024-04-17T07:55:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.438+0000 7fcb43e00640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb300a3bb0 0x7fcb300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.438+0000 7fcb43e00640 1 -- 172.21.15.46:0/4129149700 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcb440cfe80 con 0x7fcb4c105dd0 2024-04-17T07:55:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.438+0000 7fcb4b400640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb300a3bb0 0x7fcb300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.439+0000 7fcb4b400640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb300a3bb0 0x7fcb300a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009830 tx=0x7fcb3c034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:11.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.442+0000 7fcb43e00640 1 -- 172.21.15.46:0/4129149700 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcb4409cc60 con 0x7fcb4c105dd0 2024-04-17T07:55:11.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:11 smithi046 ceph-mon[32038]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:11.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:11 smithi062 ceph-mon[36507]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:11.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.715+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb4c1a7310 con 0x7fcb4c105dd0 2024-04-17T07:55:11.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.716+0000 7fcb43e00640 1 -- 172.21.15.46:0/4129149700 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7fcb4409c560 con 0x7fcb4c105dd0 2024-04-17T07:55:11.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:11.716 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:55:11.716 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 10 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb300a3bb0 msgr2=0x7fcb300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb300a3bb0 0x7fcb300a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009830 tx=0x7fcb3c034ad0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb4c105dd0 msgr2=0x7fcb4c1a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb4c105dd0 0x7fcb4c1a5ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400efc0 tx=0x7fcb4400c800 comp rx=0 tx=0).stop 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 shutdown_connections 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb300a3bb0 0x7fcb300a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb4c105dd0 0x7fcb4c1a5ec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb4c105410 0x7fcb4c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 --2- 172.21.15.46:0/4129149700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb4c104220 0x7fcb4c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 >> 172.21.15.46:0/4129149700 conn(0x7fcb4c0ff7b0 msgr2=0x7fcb4c079480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 shutdown_connections 2024-04-17T07:55:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:11.719+0000 7fcb52600640 1 -- 172.21.15.46:0/4129149700 wait complete. 2024-04-17T07:55:11.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:11 smithi053 ceph-mon[36437]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:12.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:12 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4129149700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:12.474 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 07:55:12 smithi046 podman[73464]: 2024-04-17 07:55:12.292412025 +0000 UTC m=+0.200202033 container create b88ece623d38b12f41c9fc2efa212f345b5edface0aa953d78516eb6f4fac0f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-17T07:55:12.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:12 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4129149700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:12.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:12 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4129149700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:12.860 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 07:55:12 smithi046 podman[73464]: 2024-04-17 07:55:12.588411905 +0000 UTC m=+0.496201920 container init b88ece623d38b12f41c9fc2efa212f345b5edface0aa953d78516eb6f4fac0f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:55:12.860 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 07:55:12 smithi046 podman[73464]: 2024-04-17 07:55:12.600766496 +0000 UTC m=+0.508556511 container start b88ece623d38b12f41c9fc2efa212f345b5edface0aa953d78516eb6f4fac0f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-17T07:55:12.860 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 07:55:12 smithi046 ceph-mds[73484]: -- 172.21.15.46:0/3359926771 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6aa0a9a40 con 0x55e6aa07c800 2024-04-17T07:55:12.860 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 07:55:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g[73480]: starting mds.g at 2024-04-17T07:55:12.860 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 07:55:12 smithi046 bash[73464]: b88ece623d38b12f41c9fc2efa212f345b5edface0aa953d78516eb6f4fac0f7 2024-04-17T07:55:12.860 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 07:55:12 smithi046 systemd[1]: Started Ceph mds.g for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:55:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:13 smithi062 ceph-mon[36507]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:13.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:13 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:13.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:13 smithi046 ceph-mon[32038]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:13.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:13.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:13.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:13 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:13.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:13 smithi053 ceph-mon[36437]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:13.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:13.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:13.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:13.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:13 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:13.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 -- 172.21.15.46:0/2779963279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4084108500 msgr2=0x7f4084108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 --2- 172.21.15.46:0/2779963279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4084108500 0x7f4084108920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4074009900 tx=0x7f407402bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 -- 172.21.15.46:0/2779963279 shutdown_connections 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 --2- 172.21.15.46:0/2779963279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f408410a060 0x7f40841108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 --2- 172.21.15.46:0/2779963279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40841096a0 0x7f4084109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 --2- 172.21.15.46:0/2779963279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4084108500 0x7f4084108920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 -- 172.21.15.46:0/2779963279 >> 172.21.15.46:0/2779963279 conn(0x7f4084077450 msgr2=0x7f4084079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 -- 172.21.15.46:0/2779963279 shutdown_connections 2024-04-17T07:55:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.939+0000 7f4089c00640 1 -- 172.21.15.46:0/2779963279 wait complete. 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.940+0000 7f4089c00640 1 Processor -- start 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.940+0000 7f4089c00640 1 -- start start 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4089c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4084108500 0x7f408419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4089c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40841096a0 0x7f408419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4089c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f408410a060 0x7f40841a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4089c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4084114110 con 0x7f40841096a0 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4089c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4084113f90 con 0x7f4084108500 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4089c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4084114290 con 0x7f408410a060 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4083e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4084108500 0x7f408419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4083400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40841096a0 0x7f408419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4083e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4084108500 0x7f408419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32798/0 (socket says 172.21.15.46:32798) 2024-04-17T07:55:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4083e00640 1 -- 172.21.15.46:0/3881571334 learned_addr learned my addr 172.21.15.46:0/3881571334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.941+0000 7f4089200640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f408410a060 0x7f40841a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4083400640 1 -- 172.21.15.46:0/3881571334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f408410a060 msgr2=0x7f40841a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4083400640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f408410a060 0x7f40841a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4083400640 1 -- 172.21.15.46:0/3881571334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4084108500 msgr2=0x7f408419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4083400640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4084108500 0x7f408419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4083400640 1 -- 172.21.15.46:0/3881571334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40841a63d0 con 0x7f40841096a0 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4083e00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4084108500 0x7f408419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4083400640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40841096a0 0x7f408419ecf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f407000eb00 tx=0x7f407000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4080c00640 1 -- 172.21.15.46:0/3881571334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070010430 con 0x7f40841096a0 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40841a66c0 con 0x7f40841096a0 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4080c00640 1 -- 172.21.15.46:0/3881571334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4070010ab0 con 0x7f40841096a0 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4080c00640 1 -- 172.21.15.46:0/3881571334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070003b70 con 0x7f40841096a0 2024-04-17T07:55:13.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.942+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40841a6c60 con 0x7f40841096a0 2024-04-17T07:55:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.944+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408410f850 con 0x7f40841096a0 2024-04-17T07:55:13.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.946+0000 7f4080c00640 1 -- 172.21.15.46:0/3881571334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4070010c60 con 0x7f40841096a0 2024-04-17T07:55:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.948+0000 7f4080c00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f40680a3ba0 0x7f40680a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.948+0000 7f4080c00640 1 -- 172.21.15.46:0/3881571334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f40700d0900 con 0x7f40841096a0 2024-04-17T07:55:13.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.948+0000 7f4083e00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f40680a3ba0 0x7f40680a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:13.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.949+0000 7f4083e00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f40680a3ba0 0x7f40680a6060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f407402c8e0 tx=0x7f4074005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:13.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:13.952+0000 7f4080c00640 1 -- 172.21.15.46:0/3881571334 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f407009d6e0 con 0x7f40841096a0 2024-04-17T07:55:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.209+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f408402d5d0 con 0x7f40841096a0 2024-04-17T07:55:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.211+0000 7f4080c00640 1 -- 172.21.15.46:0/3881571334 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8812 (secure 0 0 0) 0x7f407009cfe0 con 0x7f40841096a0 2024-04-17T07:55:14.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:14.210 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-17T07:55:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 11 2024-04-17T07:55:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f40680a3ba0 msgr2=0x7f40680a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f40680a3ba0 0x7f40680a6060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f407402c8e0 tx=0x7f4074005b50 comp rx=0 tx=0).stop 2024-04-17T07:55:14.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40841096a0 msgr2=0x7f408419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40841096a0 0x7f408419ecf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f407000eb00 tx=0x7f407000c2d0 comp rx=0 tx=0).stop 2024-04-17T07:55:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 shutdown_connections 2024-04-17T07:55:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f40680a3ba0 0x7f40680a6060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f408410a060 0x7f40841a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40841096a0 0x7f408419ecf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:14.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 --2- 172.21.15.46:0/3881571334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4084108500 0x7f408419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.213+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 >> 172.21.15.46:0/3881571334 conn(0x7f4084077450 msgr2=0x7f408410ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.214+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 shutdown_connections 2024-04-17T07:55:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:14.214+0000 7f4089c00640 1 -- 172.21.15.46:0/3881571334 wait complete. 2024-04-17T07:55:14.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:14 smithi046 ceph-mon[32038]: Deploying daemon mds.a on smithi046 2024-04-17T07:55:14.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:14 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] up:boot 2024-04-17T07:55:14.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:14 smithi046 ceph-mon[32038]: fsmap 10 up:standby 2024-04-17T07:55:14.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:14 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T07:55:14.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:14 smithi046 ceph-mon[32038]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:14.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:14 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3881571334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:14 smithi062 ceph-mon[36507]: Deploying daemon mds.a on smithi046 2024-04-17T07:55:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:14 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] up:boot 2024-04-17T07:55:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:14 smithi062 ceph-mon[36507]: fsmap 10 up:standby 2024-04-17T07:55:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:14 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T07:55:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:14 smithi062 ceph-mon[36507]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:14.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:14 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3881571334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:14.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:14 smithi053 ceph-mon[36437]: Deploying daemon mds.a on smithi046 2024-04-17T07:55:14.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:14 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] up:boot 2024-04-17T07:55:14.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:14 smithi053 ceph-mon[36437]: fsmap 10 up:standby 2024-04-17T07:55:14.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:14 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T07:55:14.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:14 smithi053 ceph-mon[36437]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:14.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:14 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3881571334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:16.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:16.306 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 systemd[1]: Starting Ceph mds.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.437+0000 7f905fe00640 1 -- 172.21.15.46:0/1952444315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 msgr2=0x7f90600743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.437+0000 7f905fe00640 1 --2- 172.21.15.46:0/1952444315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 0x7f90600743f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9050009cd0 tx=0x7f905002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.437+0000 7f905fe00640 1 -- 172.21.15.46:0/1952444315 shutdown_connections 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.437+0000 7f905fe00640 1 --2- 172.21.15.46:0/1952444315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906010e480 0x7f9060110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.437+0000 7f905fe00640 1 --2- 172.21.15.46:0/1952444315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 0x7f9060074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.437+0000 7f905fe00640 1 --2- 172.21.15.46:0/1952444315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 0x7f90600743f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.437+0000 7f905fe00640 1 -- 172.21.15.46:0/1952444315 >> 172.21.15.46:0/1952444315 conn(0x7f90600fd420 msgr2=0x7f90600ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.438+0000 7f905fe00640 1 -- 172.21.15.46:0/1952444315 shutdown_connections 2024-04-17T07:55:16.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.438+0000 7f905fe00640 1 -- 172.21.15.46:0/1952444315 wait complete. 2024-04-17T07:55:16.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 Processor -- start 2024-04-17T07:55:16.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 -- start start 2024-04-17T07:55:16.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 0x7f906019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 0x7f906019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906010e480 0x7f90601a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9060114140 con 0x7f90600749c0 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9060113fc0 con 0x7f9060075fa0 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90601142c0 con 0x7f906010e480 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.439+0000 7f905ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 0x7f906019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 0x7f906019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34830/0 (socket says 172.21.15.46:34830) 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 -- 172.21.15.46:0/268352648 learned_addr learned my addr 172.21.15.46:0/268352648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905f400640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906010e480 0x7f90601a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f9056000640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 0x7f906019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 -- 172.21.15.46:0/268352648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906010e480 msgr2=0x7f90601a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906010e480 0x7f90601a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 -- 172.21.15.46:0/268352648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 msgr2=0x7f906019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:16.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 0x7f906019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 -- 172.21.15.46:0/268352648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90601a6410 con 0x7f90600749c0 2024-04-17T07:55:16.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.440+0000 7f905ea00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 0x7f906019e7f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f905002c700 tx=0x7f9050032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:16.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.441+0000 7f9057e00640 1 -- 172.21.15.46:0/268352648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050041020 con 0x7f90600749c0 2024-04-17T07:55:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.441+0000 7f9057e00640 1 -- 172.21.15.46:0/268352648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9050034cd0 con 0x7f90600749c0 2024-04-17T07:55:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.441+0000 7f9057e00640 1 -- 172.21.15.46:0/268352648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050039040 con 0x7f90600749c0 2024-04-17T07:55:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.441+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90601a66a0 con 0x7f90600749c0 2024-04-17T07:55:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.441+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90601a6b40 con 0x7f90600749c0 2024-04-17T07:55:16.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.442+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9060104650 con 0x7f90600749c0 2024-04-17T07:55:16.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.444+0000 7f9057e00640 1 -- 172.21.15.46:0/268352648 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9050044020 con 0x7f90600749c0 2024-04-17T07:55:16.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.445+0000 7f9057e00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90440a3c20 0x7f90440a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.446+0000 7f9057e00640 1 -- 172.21.15.46:0/268352648 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f90500f1e40 con 0x7f90600749c0 2024-04-17T07:55:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.446+0000 7f9056000640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90440a3c20 0x7f90440a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:16.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.447+0000 7f9056000640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90440a3c20 0x7f90440a60e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f906019fd00 tx=0x7f9048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.451+0000 7f9057e00640 1 -- 172.21.15.46:0/268352648 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f905004b020 con 0x7f90600749c0 2024-04-17T07:55:16.564 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:16 smithi046 ceph-mon[32038]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:16.564 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 podman[73810]: 2024-04-17 07:55:16.283430807 +0000 UTC m=+0.034809194 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:55:16.564 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 podman[73810]: 2024-04-17 07:55:16.399705633 +0000 UTC m=+0.151084063 container create bbe75030bc0aadd83f82c2df6a111d63fbd475aae88eaa5024a48081e1183c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-17T07:55:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.705+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f906002d5d0 con 0x7f90600749c0 2024-04-17T07:55:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.706+0000 7f9057e00640 1 -- 172.21.15.46:0/268352648 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8812 (secure 0 0 0) 0x7f90500becb0 con 0x7f90600749c0 2024-04-17T07:55:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:16.706 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-17T07:55:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 11 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90440a3c20 msgr2=0x7f90440a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90440a3c20 0x7f90440a60e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f906019fd00 tx=0x7f9048005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 msgr2=0x7f906019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 0x7f906019e7f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f905002c700 tx=0x7f9050032f40 comp rx=0 tx=0).stop 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 shutdown_connections 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90440a3c20 0x7f90440a60e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906010e480 0x7f90601a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9060075fa0 0x7f906019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 --2- 172.21.15.46:0/268352648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90600749c0 0x7f906019e7f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.709+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 >> 172.21.15.46:0/268352648 conn(0x7f90600fd420 msgr2=0x7f9060101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.710+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 shutdown_connections 2024-04-17T07:55:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:16.710+0000 7f905fe00640 1 -- 172.21.15.46:0/268352648 wait complete. 2024-04-17T07:55:16.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:16 smithi062 ceph-mon[36507]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:16.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:16 smithi053 ceph-mon[36437]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:17.110 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 podman[73810]: 2024-04-17 07:55:16.707927321 +0000 UTC m=+0.459305727 container init bbe75030bc0aadd83f82c2df6a111d63fbd475aae88eaa5024a48081e1183c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:55:17.110 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 podman[73810]: 2024-04-17 07:55:16.717441032 +0000 UTC m=+0.468819442 container start bbe75030bc0aadd83f82c2df6a111d63fbd475aae88eaa5024a48081e1183c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-17T07:55:17.110 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 ceph-mds[73870]: -- 172.21.15.46:0/196543233 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e8be1fa40 con 0x557e8bdf2c00 2024-04-17T07:55:17.110 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a[73864]: starting mds.a at 2024-04-17T07:55:17.111 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 bash[73810]: bbe75030bc0aadd83f82c2df6a111d63fbd475aae88eaa5024a48081e1183c8d 2024-04-17T07:55:17.111 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 07:55:16 smithi046 systemd[1]: Started Ceph mds.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:55:17.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/268352648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:17.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:17.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:17.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:17.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:17.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:17.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:17.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:17 smithi053 ceph-mon[36437]: Deploying daemon mds.b on smithi053 2024-04-17T07:55:17.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/268352648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:17.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:17.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:17.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:17.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:17.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:17.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:17.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:17 smithi046 ceph-mon[32038]: Deploying daemon mds.b on smithi053 2024-04-17T07:55:18.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/268352648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:18.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:18.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:18.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:18.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:55:18.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:55:18.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:18.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:17 smithi062 ceph-mon[36507]: Deploying daemon mds.b on smithi053 2024-04-17T07:55:18.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:18 smithi053 ceph-mon[36437]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:18.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:18 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.46:6840/1599229052,v1:172.21.15.46:6841/1599229052] up:boot 2024-04-17T07:55:18.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:18 smithi053 ceph-mon[36437]: fsmap 11 up:standby 2024-04-17T07:55:18.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:18 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T07:55:18.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:18.823 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:18 smithi046 ceph-mon[32038]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:18.823 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:18 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.46:6840/1599229052,v1:172.21.15.46:6841/1599229052] up:boot 2024-04-17T07:55:18.824 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:18 smithi046 ceph-mon[32038]: fsmap 11 up:standby 2024-04-17T07:55:18.824 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:18 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T07:55:18.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.966+0000 7f4087200640 1 -- 172.21.15.46:0/742371851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103040 msgr2=0x7f40881034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:18.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.966+0000 7f4087200640 1 --2- 172.21.15.46:0/742371851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103040 0x7f40881034c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4074009900 tx=0x7f407402bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:18.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.966+0000 7f4087200640 1 -- 172.21.15.46:0/742371851 shutdown_connections 2024-04-17T07:55:18.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.966+0000 7f4087200640 1 --2- 172.21.15.46:0/742371851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088103a00 0x7f408810a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:18.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.966+0000 7f4087200640 1 --2- 172.21.15.46:0/742371851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103040 0x7f40881034c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:18.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.966+0000 7f4087200640 1 --2- 172.21.15.46:0/742371851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4088101e60 0x7f4088102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:18.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.966+0000 7f4087200640 1 -- 172.21.15.46:0/742371851 >> 172.21.15.46:0/742371851 conn(0x7f40880fd420 msgr2=0x7f40880ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:18.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.967+0000 7f4087200640 1 -- 172.21.15.46:0/742371851 shutdown_connections 2024-04-17T07:55:18.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.967+0000 7f4087200640 1 -- 172.21.15.46:0/742371851 wait complete. 2024-04-17T07:55:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 Processor -- start 2024-04-17T07:55:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 -- start start 2024-04-17T07:55:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088101e60 0x7f408819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4088103040 0x7f408819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103a00 0x7f40881a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f408810dbb0 con 0x7f4088101e60 2024-04-17T07:55:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f408810da30 con 0x7f4088103a00 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.968+0000 7f4087200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f408810dd30 con 0x7f4088103040 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4085e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088101e60 0x7f408819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103a00 0x7f40881a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103a00 0x7f40881a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40384/0 (socket says 172.21.15.46:40384) 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 -- 172.21.15.46:0/755312222 learned_addr learned my addr 172.21.15.46:0/755312222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4085e00640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088101e60 0x7f408819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37272/0 (socket says 172.21.15.46:37272) 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4085400640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4088103040 0x7f408819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 -- 172.21.15.46:0/755312222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4088103040 msgr2=0x7f408819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4088103040 0x7f408819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 -- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088101e60 msgr2=0x7f408819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088101e60 0x7f408819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4086800640 1 -- 172.21.15.46:0/755312222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40881a2070 con 0x7f4088103a00 2024-04-17T07:55:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.969+0000 7f4085e00640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088101e60 0x7f408819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.970+0000 7f4086800640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103a00 0x7f40881a1a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f408000e9b0 tx=0x7f408000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.970+0000 7f407ea00640 1 -- 172.21.15.46:0/755312222 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408000cd80 con 0x7f4088103a00 2024-04-17T07:55:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.970+0000 7f407ea00640 1 -- 172.21.15.46:0/755312222 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40800035f0 con 0x7f4088103a00 2024-04-17T07:55:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.970+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40881a2360 con 0x7f4088103a00 2024-04-17T07:55:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.970+0000 7f407ea00640 1 -- 172.21.15.46:0/755312222 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080010670 con 0x7f4088103a00 2024-04-17T07:55:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.970+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40881a2830 con 0x7f4088103a00 2024-04-17T07:55:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.971+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40881a2af0 con 0x7f4088103a00 2024-04-17T07:55:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.973+0000 7f407ea00640 1 -- 172.21.15.46:0/755312222 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f40800037a0 con 0x7f4088103a00 2024-04-17T07:55:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.975+0000 7f407ea00640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f406c0a3ba0 0x7f406c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.975+0000 7f407ea00640 1 -- 172.21.15.46:0/755312222 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f40800d0350 con 0x7f4088103a00 2024-04-17T07:55:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.975+0000 7f4085e00640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f406c0a3ba0 0x7f406c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.976+0000 7f4085e00640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f406c0a3ba0 0x7f406c0a6060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4088102bb0 tx=0x7f4078009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:18.977+0000 7f407ea00640 1 -- 172.21.15.46:0/755312222 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f408009d1c0 con 0x7f4088103a00 2024-04-17T07:55:19.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:18 smithi062 ceph-mon[36507]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:19.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:18 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.46:6840/1599229052,v1:172.21.15.46:6841/1599229052] up:boot 2024-04-17T07:55:19.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:18 smithi062 ceph-mon[36507]: fsmap 11 up:standby 2024-04-17T07:55:19.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:18 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T07:55:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.224+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40881a2cc0 con 0x7f4088103a00 2024-04-17T07:55:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.226+0000 7f407ea00640 1 -- 172.21.15.46:0/755312222 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9638 (secure 0 0 0) 0x7f408009cac0 con 0x7f4088103a00 2024-04-17T07:55:19.225 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:19.225 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-17T07:55:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 12 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.228+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f406c0a3ba0 msgr2=0x7f406c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.228+0000 7f4087200640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f406c0a3ba0 0x7f406c0a6060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4088102bb0 tx=0x7f4078009500 comp rx=0 tx=0).stop 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.228+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103a00 msgr2=0x7f40881a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.228+0000 7f4087200640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103a00 0x7f40881a1a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f408000e9b0 tx=0x7f408000ee70 comp rx=0 tx=0).stop 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.228+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 shutdown_connections 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.229+0000 7f4087200640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f406c0a3ba0 0x7f406c0a6060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.229+0000 7f4087200640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4088103a00 0x7f40881a1a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.229+0000 7f4087200640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4088103040 0x7f408819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.229+0000 7f4087200640 1 --2- 172.21.15.46:0/755312222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4088101e60 0x7f408819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.229+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 >> 172.21.15.46:0/755312222 conn(0x7f40880fd420 msgr2=0x7f4088074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.229+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 shutdown_connections 2024-04-17T07:55:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:19.229+0000 7f4087200640 1 -- 172.21.15.46:0/755312222 wait complete. 2024-04-17T07:55:19.471 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:19 smithi053 systemd[1]: Starting Ceph mds.b for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T07:55:19.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:19 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/755312222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:19.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:19 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/755312222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:19.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:19 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/755312222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:19.971 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:19 smithi053 podman[67378]: 2024-04-17 07:55:19.507689724 +0000 UTC m=+0.018183084 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:55:19.971 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:19 smithi053 podman[67378]: 2024-04-17 07:55:19.621927122 +0000 UTC m=+0.132420521 container create 98169a4dcdb2cc5f13a834a68b79630f75b4a35d46cf903989a2b91e31d82e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-17T07:55:20.471 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:20 smithi053 podman[67378]: 2024-04-17 07:55:20.010113719 +0000 UTC m=+0.520607079 container init 98169a4dcdb2cc5f13a834a68b79630f75b4a35d46cf903989a2b91e31d82e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-17T07:55:20.471 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:20 smithi053 podman[67378]: 2024-04-17 07:55:20.013100079 +0000 UTC m=+0.523593442 container start 98169a4dcdb2cc5f13a834a68b79630f75b4a35d46cf903989a2b91e31d82e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:55:20.471 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:20 smithi053 ceph-mds[67397]: -- 172.21.15.53:0/2081385601 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a28f41a40 con 0x561a28f15000 2024-04-17T07:55:20.471 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: starting mds.b at 2024-04-17T07:55:20.471 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:20 smithi053 bash[67378]: 98169a4dcdb2cc5f13a834a68b79630f75b4a35d46cf903989a2b91e31d82e27 2024-04-17T07:55:20.471 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:20 smithi053 systemd[1]: Started Ceph mds.b for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T07:55:21.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:21.436 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:21.436 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:21.436 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:21.436 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:21.436 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:21.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:21.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:55:21.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] up:boot 2024-04-17T07:55:21.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: fsmap 12 up:standby 2024-04-17T07:55:21.437 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T07:55:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 -- 172.21.15.46:0/3822402621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c078500 msgr2=0x7f5e8c078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 --2- 172.21.15.46:0/3822402621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c078500 0x7f5e8c078da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009860 tx=0x7f5e8402bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 -- 172.21.15.46:0/3822402621 shutdown_connections 2024-04-17T07:55:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 --2- 172.21.15.46:0/3822402621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c078500 0x7f5e8c078da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 --2- 172.21.15.46:0/3822402621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e8c077b60 0x7f5e8c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 --2- 172.21.15.46:0/3822402621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c076920 0x7f5e8c076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 -- 172.21.15.46:0/3822402621 >> 172.21.15.46:0/3822402621 conn(0x7f5e8c0ff7b0 msgr2=0x7f5e8c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 -- 172.21.15.46:0/3822402621 shutdown_connections 2024-04-17T07:55:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.450+0000 7f5e93a00640 1 -- 172.21.15.46:0/3822402621 wait complete. 2024-04-17T07:55:21.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.451+0000 7f5e93a00640 1 Processor -- start 2024-04-17T07:55:21.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.451+0000 7f5e93a00640 1 -- start start 2024-04-17T07:55:21.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.451+0000 7f5e93a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c076920 0x7f5e8c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:21.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.451+0000 7f5e93a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c077b60 0x7f5e8c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.451+0000 7f5e93a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e8c078500 0x7f5e8c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.451+0000 7f5e93a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c114260 con 0x7f5e8c076920 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.451+0000 7f5e93a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1140e0 con 0x7f5e8c077b60 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e93a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1143e0 con 0x7f5e8c078500 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c077b60 0x7f5e8c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e91000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c076920 0x7f5e8c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c077b60 0x7f5e8c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40404/0 (socket says 172.21.15.46:40404) 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 -- 172.21.15.46:0/4198334628 learned_addr learned my addr 172.21.15.46:0/4198334628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e91a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e8c078500 0x7f5e8c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 -- 172.21.15.46:0/4198334628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e8c078500 msgr2=0x7f5e8c1a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e8c078500 0x7f5e8c1a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 -- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c076920 msgr2=0x7f5e8c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c076920 0x7f5e8c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e8be00640 1 -- 172.21.15.46:0/4198334628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c1a6450 con 0x7f5e8c077b60 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.452+0000 7f5e91a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e8c078500 0x7f5e8c1a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.453+0000 7f5e8be00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c077b60 0x7f5e8c19ed70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400efb0 tx=0x7f5e7400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.453+0000 7f5e89600640 1 -- 172.21.15.46:0/4198334628 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7401b070 con 0x7f5e8c077b60 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.453+0000 7f5e89600640 1 -- 172.21.15.46:0/4198334628 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7400ba50 con 0x7f5e8c077b60 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.453+0000 7f5e89600640 1 -- 172.21.15.46:0/4198334628 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74016c00 con 0x7f5e8c077b60 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.453+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c1a6740 con 0x7f5e8c077b60 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.453+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a6ce0 con 0x7f5e8c077b60 2024-04-17T07:55:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.454+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c110cf0 con 0x7f5e8c077b60 2024-04-17T07:55:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.456+0000 7f5e89600640 1 -- 172.21.15.46:0/4198334628 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e7401e070 con 0x7f5e8c077b60 2024-04-17T07:55:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.457+0000 7f5e89600640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5e700a3c30 0x7f5e700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.458+0000 7f5e89600640 1 -- 172.21.15.46:0/4198334628 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5e740cfea0 con 0x7f5e8c077b60 2024-04-17T07:55:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.458+0000 7f5e91000640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5e700a3c30 0x7f5e700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.458+0000 7f5e91000640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5e700a3c30 0x7f5e700a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009bd0 tx=0x7f5e7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:21.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.462+0000 7f5e89600640 1 -- 172.21.15.46:0/4198334628 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5e7409c630 con 0x7f5e8c077b60 2024-04-17T07:55:21.522 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:22.077 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] up:boot 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: fsmap 12 up:standby 2024-04-17T07:55:22.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T07:55:22.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:22.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:55:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] up:boot 2024-04-17T07:55:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: fsmap 12 up:standby 2024-04-17T07:55:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T07:55:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.722+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e8c02d5d0 con 0x7f5e8c077b60 2024-04-17T07:55:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.723+0000 7f5e89600640 1 -- 172.21.15.46:0/4198334628 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10461 (secure 0 0 0) 0x7f5e740a0930 con 0x7f5e8c077b60 2024-04-17T07:55:22.081 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:22.081 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5e700a3c30 msgr2=0x7f5e700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5e700a3c30 0x7f5e700a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009bd0 tx=0x7f5e7c005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c077b60 msgr2=0x7f5e8c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c077b60 0x7f5e8c19ed70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400efb0 tx=0x7f5e7400c800 comp rx=0 tx=0).stop 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 shutdown_connections 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5e700a3c30 0x7f5e700a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e8c078500 0x7f5e8c1a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c077b60 0x7f5e8c19ed70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 --2- 172.21.15.46:0/4198334628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c076920 0x7f5e8c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 >> 172.21.15.46:0/4198334628 conn(0x7f5e8c0ff7b0 msgr2=0x7f5e8c100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 shutdown_connections 2024-04-17T07:55:22.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:21.727+0000 7f5e93a00640 1 -- 172.21.15.46:0/4198334628 wait complete. 2024-04-17T07:55:22.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.319+0000 7fe251600640 1 -- 172.21.15.46:0/3495004214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c105c70 msgr2=0x7fe24c10c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.319+0000 7fe251600640 1 --2- 172.21.15.46:0/3495004214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c105c70 0x7fe24c10c500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe244009ca0 tx=0x7fe24402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.319+0000 7fe251600640 1 -- 172.21.15.46:0/3495004214 shutdown_connections 2024-04-17T07:55:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.319+0000 7fe251600640 1 --2- 172.21.15.46:0/3495004214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c105c70 0x7fe24c10c500 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.319+0000 7fe251600640 1 --2- 172.21.15.46:0/3495004214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1052b0 0x7fe24c105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.319+0000 7fe251600640 1 --2- 172.21.15.46:0/3495004214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c1040c0 0x7fe24c1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.319+0000 7fe251600640 1 -- 172.21.15.46:0/3495004214 >> 172.21.15.46:0/3495004214 conn(0x7fe24c0ff630 msgr2=0x7fe24c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.320+0000 7fe251600640 1 -- 172.21.15.46:0/3495004214 shutdown_connections 2024-04-17T07:55:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.320+0000 7fe251600640 1 -- 172.21.15.46:0/3495004214 wait complete. 2024-04-17T07:55:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 Processor -- start 2024-04-17T07:55:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 -- start start 2024-04-17T07:55:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1040c0 0x7fe24c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c1052b0 0x7fe24c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c105c70 0x7fe24c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c10fd50 con 0x7fe24c1052b0 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c10fbd0 con 0x7fe24c1040c0 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe251600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c10fed0 con 0x7fe24c105c70 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe250c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c105c70 0x7fe24c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe24be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1040c0 0x7fe24c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe24be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1040c0 0x7fe24c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40412/0 (socket says 172.21.15.46:40412) 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe250c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c105c70 0x7fe24c1a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:45130/0 (socket says 172.21.15.46:45130) 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe24be00640 1 -- 172.21.15.46:0/1093063836 learned_addr learned my addr 172.21.15.46:0/1093063836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.321+0000 7fe24b400640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c1052b0 0x7fe24c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe250c00640 1 -- 172.21.15.46:0/1093063836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1040c0 msgr2=0x7fe24c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe250c00640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1040c0 0x7fe24c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe250c00640 1 -- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c1052b0 msgr2=0x7fe24c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe250c00640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c1052b0 0x7fe24c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe250c00640 1 -- 172.21.15.46:0/1093063836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c1a1fa0 con 0x7fe24c105c70 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe24be00640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1040c0 0x7fe24c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe250c00640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c105c70 0x7fe24c1a1930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe24402c630 tx=0x7fe244032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe248c00640 1 -- 172.21.15.46:0/1093063836 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244041020 con 0x7fe24c105c70 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe248c00640 1 -- 172.21.15.46:0/1093063836 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe244034cd0 con 0x7fe24c105c70 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.322+0000 7fe248c00640 1 -- 172.21.15.46:0/1093063836 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244039040 con 0x7fe24c105c70 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.323+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c1a2230 con 0x7fe24c105c70 2024-04-17T07:55:22.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.323+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c1a26a0 con 0x7fe24c105c70 2024-04-17T07:55:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.324+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24c10b310 con 0x7fe24c105c70 2024-04-17T07:55:22.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.326+0000 7fe248c00640 1 -- 172.21.15.46:0/1093063836 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe244044020 con 0x7fe24c105c70 2024-04-17T07:55:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.327+0000 7fe248c00640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe2300a3c20 0x7fe2300a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.327+0000 7fe248c00640 1 -- 172.21.15.46:0/1093063836 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe2440f1cf0 con 0x7fe24c105c70 2024-04-17T07:55:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.327+0000 7fe24be00640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe2300a3c20 0x7fe2300a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.328+0000 7fe24be00640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe2300a3c20 0x7fe2300a60e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0058c0 tx=0x7fe23c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:22.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.331+0000 7fe248c00640 1 -- 172.21.15.46:0/1093063836 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe24404b020 con 0x7fe24c105c70 2024-04-17T07:55:22.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.567+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe24c062b60 con 0x7fe24c105c70 2024-04-17T07:55:22.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.568+0000 7fe248c00640 1 -- 172.21.15.46:0/1093063836 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18319 (secure 0 0 0) 0x7fe2440beb60 con 0x7fe24c105c70 2024-04-17T07:55:22.568 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:22.568 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":81,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:54:27.818161+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:22.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe2300a3c20 msgr2=0x7fe2300a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe2300a3c20 0x7fe2300a60e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0058c0 tx=0x7fe23c005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:22.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c105c70 msgr2=0x7fe24c1a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c105c70 0x7fe24c1a1930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe24402c630 tx=0x7fe244032f20 comp rx=0 tx=0).stop 2024-04-17T07:55:22.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 shutdown_connections 2024-04-17T07:55:22.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe2300a3c20 0x7fe2300a60e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe24c105c70 0x7fe24c1a1930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c1052b0 0x7fe24c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 --2- 172.21.15.46:0/1093063836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe24c1040c0 0x7fe24c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 >> 172.21.15.46:0/1093063836 conn(0x7fe24c0ff630 msgr2=0x7fe24c1084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:22.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 shutdown_connections 2024-04-17T07:55:22.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.571+0000 7fe251600640 1 -- 172.21.15.46:0/1093063836 wait complete. 2024-04-17T07:55:22.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T07:55:22.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:55:22.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:22 smithi046 ceph-mon[32038]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:22.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:22 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4198334628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:22.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.801+0000 7f76a5000640 1 -- 172.21.15.46:0/4080216567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0102fc0 msgr2=0x7f76a0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.801+0000 7f76a5000640 1 --2- 172.21.15.46:0/4080216567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0102fc0 0x7f76a0103440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f768c009900 tx=0x7f768c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.802+0000 7f76a5000640 1 -- 172.21.15.46:0/4080216567 shutdown_connections 2024-04-17T07:55:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.802+0000 7f76a5000640 1 --2- 172.21.15.46:0/4080216567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0103980 0x7f76a010a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.802+0000 7f76a5000640 1 --2- 172.21.15.46:0/4080216567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0102fc0 0x7f76a0103440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.802+0000 7f76a5000640 1 --2- 172.21.15.46:0/4080216567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0101dd0 0x7f76a01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.802+0000 7f76a5000640 1 -- 172.21.15.46:0/4080216567 >> 172.21.15.46:0/4080216567 conn(0x7f76a00fd340 msgr2=0x7f76a00ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.802+0000 7f76a5000640 1 -- 172.21.15.46:0/4080216567 shutdown_connections 2024-04-17T07:55:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.802+0000 7f76a5000640 1 -- 172.21.15.46:0/4080216567 wait complete. 2024-04-17T07:55:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.803+0000 7f76a5000640 1 Processor -- start 2024-04-17T07:55:22.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:22 smithi062 ceph-mon[36507]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:22.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:22 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4198334628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:22.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.803+0000 7f76a5000640 1 -- start start 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f76a5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0101dd0 0x7f76a019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f76a5000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0102fc0 0x7f76a019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f76a5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0103980 0x7f76a01a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f76a5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76a010daa0 con 0x7f76a0103980 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f76a5000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76a010d920 con 0x7f76a0102fc0 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f76a5000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76a010dc20 con 0x7f76a0101dd0 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0102fc0 0x7f76a019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0103980 0x7f76a01a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0101dd0 0x7f76a019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0102fc0 0x7f76a019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40424/0 (socket says 172.21.15.46:40424) 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769ea00640 1 -- 172.21.15.46:0/1936960947 learned_addr learned my addr 172.21.15.46:0/1936960947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769ea00640 1 -- 172.21.15.46:0/1936960947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0101dd0 msgr2=0x7f76a019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769ea00640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0101dd0 0x7f76a019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769ea00640 1 -- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0103980 msgr2=0x7f76a01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.804+0000 7f769ea00640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0103980 0x7f76a01a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f769ea00640 1 -- 172.21.15.46:0/1936960947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a01a1f60 con 0x7f76a0102fc0 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f769f400640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0101dd0 0x7f76a019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f769ea00640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0102fc0 0x7f76a019a8d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f768c005ab0 tx=0x7f768c02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f7697e00640 1 -- 172.21.15.46:0/1936960947 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c043020 con 0x7f76a0102fc0 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f7697e00640 1 -- 172.21.15.46:0/1936960947 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768c0054e0 con 0x7f76a0102fc0 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f7697e00640 1 -- 172.21.15.46:0/1936960947 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c003ce0 con 0x7f76a0102fc0 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a01a21f0 con 0x7f76a0102fc0 2024-04-17T07:55:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.805+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a01a26f0 con 0x7f76a0102fc0 2024-04-17T07:55:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.807+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a01a28f0 con 0x7f76a0102fc0 2024-04-17T07:55:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.808+0000 7f7697e00640 1 -- 172.21.15.46:0/1936960947 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f768c046020 con 0x7f76a0102fc0 2024-04-17T07:55:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.810+0000 7f7697e00640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76840a3c20 0x7f76840a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.810+0000 7f7697e00640 1 -- 172.21.15.46:0/1936960947 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f768c0f1fe0 con 0x7f76a0102fc0 2024-04-17T07:55:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.810+0000 7f769f400640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76840a3c20 0x7f76840a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.811+0000 7f769f400640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76840a3c20 0x7f76840a60e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76a0102e20 tx=0x7f7690005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:22.814+0000 7f7697e00640 1 -- 172.21.15.46:0/1936960947 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f768c04d020 con 0x7f76a0102fc0 2024-04-17T07:55:22.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:22 smithi053 ceph-mon[36437]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:22.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:22 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4198334628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:22.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:22.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.080+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76a01a2ac0 con 0x7f76a0102fc0 2024-04-17T07:55:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.081+0000 7f7697e00640 1 -- 172.21.15.46:0/1936960947 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f768c0bee00 con 0x7f76a0102fc0 2024-04-17T07:55:23.080 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:55:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76840a3c20 msgr2=0x7f76840a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76840a3c20 0x7f76840a60e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76a0102e20 tx=0x7f7690005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0102fc0 msgr2=0x7f76a019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0102fc0 0x7f76a019a8d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f768c005ab0 tx=0x7f768c02c6c0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 shutdown_connections 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76840a3c20 0x7f76840a60e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a0103980 0x7f76a01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0102fc0 0x7f76a019a8d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 --2- 172.21.15.46:0/1936960947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76a0101dd0 0x7f76a019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 >> 172.21.15.46:0/1936960947 conn(0x7f76a00fd340 msgr2=0x7f76a0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 shutdown_connections 2024-04-17T07:55:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.084+0000 7f76a5000640 1 -- 172.21.15.46:0/1936960947 wait complete. 2024-04-17T07:55:23.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T07:55:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 -- 172.21.15.46:0/1137224174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4103200 msgr2=0x7f6bd4105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 --2- 172.21.15.46:0/1137224174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4103200 0x7f6bd4105670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009860 tx=0x7f6bc002bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 -- 172.21.15.46:0/1137224174 shutdown_connections 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 --2- 172.21.15.46:0/1137224174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4105bb0 0x7f6bd4106030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 --2- 172.21.15.46:0/1137224174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4103200 0x7f6bd4105670 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 --2- 172.21.15.46:0/1137224174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bd4100890 0x7f6bd4102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 -- 172.21.15.46:0/1137224174 >> 172.21.15.46:0/1137224174 conn(0x7f6bd40fc4d0 msgr2=0x7f6bd40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 -- 172.21.15.46:0/1137224174 shutdown_connections 2024-04-17T07:55:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.303+0000 7f6bdb600640 1 -- 172.21.15.46:0/1137224174 wait complete. 2024-04-17T07:55:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.304+0000 7f6bdb600640 1 Processor -- start 2024-04-17T07:55:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.305+0000 7f6bdb600640 1 -- start start 2024-04-17T07:55:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.305+0000 7f6bdb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bd4100890 0x7f6bd419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.305+0000 7f6bdb600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4103200 0x7f6bd419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.305+0000 7f6bdb600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4105bb0 0x7f6bd41a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.305+0000 7f6bdb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4114210 con 0x7f6bd4100890 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.305+0000 7f6bdb600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4114090 con 0x7f6bd4105bb0 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.305+0000 7f6bdb600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4114390 con 0x7f6bd4103200 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bd4100890 0x7f6bd419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4103200 0x7f6bd419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4103200 0x7f6bd419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:45164/0 (socket says 172.21.15.46:45164) 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bd4100890 0x7f6bd419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37342/0 (socket says 172.21.15.46:37342) 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 -- 172.21.15.46:0/3966826025 learned_addr learned my addr 172.21.15.46:0/3966826025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd9600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4105bb0 0x7f6bd41a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 -- 172.21.15.46:0/3966826025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4105bb0 msgr2=0x7f6bd41a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4105bb0 0x7f6bd41a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 -- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bd4100890 msgr2=0x7f6bd419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bd4100890 0x7f6bd419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.306+0000 7f6bd3e00640 1 -- 172.21.15.46:0/3966826025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd41a6480 con 0x7f6bd4103200 2024-04-17T07:55:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.307+0000 7f6bd3e00640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4103200 0x7f6bd419edf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00349e0 tx=0x7f6bc0032450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.307+0000 7f6bd1600640 1 -- 172.21.15.46:0/3966826025 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0041030 con 0x7f6bd4103200 2024-04-17T07:55:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.307+0000 7f6bd1600640 1 -- 172.21.15.46:0/3966826025 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc002c510 con 0x7f6bd4103200 2024-04-17T07:55:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.307+0000 7f6bd1600640 1 -- 172.21.15.46:0/3966826025 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0039040 con 0x7f6bd4103200 2024-04-17T07:55:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.307+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd41a6710 con 0x7f6bd4103200 2024-04-17T07:55:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.307+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd41a6c50 con 0x7f6bd4103200 2024-04-17T07:55:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.309+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd40fdfc0 con 0x7f6bd4103200 2024-04-17T07:55:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.311+0000 7f6bd1600640 1 -- 172.21.15.46:0/3966826025 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6bc0044070 con 0x7f6bd4103200 2024-04-17T07:55:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.312+0000 7f6bd1600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6bbc0a3c30 0x7f6bbc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.313+0000 7f6bd1600640 1 -- 172.21.15.46:0/3966826025 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6bc00f14a0 con 0x7f6bd4103200 2024-04-17T07:55:23.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.313+0000 7f6bd8c00640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6bbc0a3c30 0x7f6bbc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.314+0000 7f6bd8c00640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6bbc0a3c30 0x7f6bbc0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009bd0 tx=0x7f6bc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:23.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.317+0000 7f6bd1600640 1 -- 172.21.15.46:0/3966826025 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6bc00be280 con 0x7f6bd4103200 2024-04-17T07:55:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.551+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6bd4062b60 con 0x7f6bd4103200 2024-04-17T07:55:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.552+0000 7f6bd1600640 1 -- 172.21.15.46:0/3966826025 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f6bc00bdb80 con 0x7f6bd4103200 2024-04-17T07:55:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:23.552 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-17T07:55:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T07:55:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-17T07:55:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-17T07:55:23.552 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:55:23.552 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-17T07:55:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6bbc0a3c30 msgr2=0x7f6bbc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6bbc0a3c30 0x7f6bbc0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009bd0 tx=0x7f6bc8005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4103200 msgr2=0x7f6bd419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4103200 0x7f6bd419edf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00349e0 tx=0x7f6bc0032450 comp rx=0 tx=0).stop 2024-04-17T07:55:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 shutdown_connections 2024-04-17T07:55:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6bbc0a3c30 0x7f6bbc0a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd4105bb0 0x7f6bd41a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bd4103200 0x7f6bd419edf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 --2- 172.21.15.46:0/3966826025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bd4100890 0x7f6bd419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 >> 172.21.15.46:0/3966826025 conn(0x7f6bd40fc4d0 msgr2=0x7f6bd4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 shutdown_connections 2024-04-17T07:55:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.555+0000 7f6bdb600640 1 -- 172.21.15.46:0/3966826025 wait complete. 2024-04-17T07:55:23.573 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T07:55:23.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T07:55:23.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:23 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1093063836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:23.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:23 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1936960947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:23.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:23 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:23 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:23 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:23.768 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:23 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:55:23.769 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:23 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 -- 172.21.15.46:0/3117316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100640 msgr2=0x7f9da8100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 --2- 172.21.15.46:0/3117316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100640 0x7f9da8100a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009860 tx=0x7f9d9c02bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 -- 172.21.15.46:0/3117316974 shutdown_connections 2024-04-17T07:55:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 --2- 172.21.15.46:0/3117316974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9da8101940 0x7f9da8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 --2- 172.21.15.46:0/3117316974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8100fa0 0x7f9da8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 --2- 172.21.15.46:0/3117316974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100640 0x7f9da8100a60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 -- 172.21.15.46:0/3117316974 >> 172.21.15.46:0/3117316974 conn(0x7f9da80fc240 msgr2=0x7f9da80fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 -- 172.21.15.46:0/3117316974 shutdown_connections 2024-04-17T07:55:23.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.801+0000 7f9dae400640 1 -- 172.21.15.46:0/3117316974 wait complete. 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.802+0000 7f9dae400640 1 Processor -- start 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dae400640 1 -- start start 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dae400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9da8100640 0x7f9da819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dae400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100fa0 0x7f9da819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dae400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8101940 0x7f9da81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dae400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9da810fc70 con 0x7f9da8100640 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dae400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9da810faf0 con 0x7f9da8101940 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dae400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9da810fdf0 con 0x7f9da8100fa0 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9da8100640 0x7f9da819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dada00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8101940 0x7f9da81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dada00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8101940 0x7f9da81a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40456/0 (socket says 172.21.15.46:40456) 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9dada00640 1 -- 172.21.15.46:0/2698313327 learned_addr learned my addr 172.21.15.46:0/2698313327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.803+0000 7f9da7e00640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100fa0 0x7f9da819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dada00640 1 -- 172.21.15.46:0/2698313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100fa0 msgr2=0x7f9da819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dada00640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100fa0 0x7f9da819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dada00640 1 -- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9da8100640 msgr2=0x7f9da819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dada00640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9da8100640 0x7f9da819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dada00640 1 -- 172.21.15.46:0/2698313327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da81a1ed0 con 0x7f9da8101940 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dad000640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9da8100640 0x7f9da819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dada00640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8101940 0x7f9da81a1860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9da000eb10 tx=0x7f9da000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9da5600640 1 -- 172.21.15.46:0/2698313327 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000ce20 con 0x7f9da8101940 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da81a21c0 con 0x7f9da8101940 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9da5600640 1 -- 172.21.15.46:0/2698313327 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da0005020 con 0x7f9da8101940 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9da5600640 1 -- 172.21.15.46:0/2698313327 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0010690 con 0x7f9da8101940 2024-04-17T07:55:23.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.804+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da81a2710 con 0x7f9da8101940 2024-04-17T07:55:23.806 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:23 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1093063836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:23.806 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:23 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1936960947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:23.806 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:23 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.806 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:23 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.806 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:23 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:23.806 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:23 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:55:23.807 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:23 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.806+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da804fcf0 con 0x7f9da8101940 2024-04-17T07:55:23.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.808+0000 7f9da5600640 1 -- 172.21.15.46:0/2698313327 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9da0005540 con 0x7f9da8101940 2024-04-17T07:55:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.809+0000 7f9da5600640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d900a3c30 0x7f9d900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.809+0000 7f9da5600640 1 -- 172.21.15.46:0/2698313327 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9da00d0520 con 0x7f9da8101940 2024-04-17T07:55:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.809+0000 7f9dad000640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d900a3c30 0x7f9d900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.810+0000 7f9dad000640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d900a3c30 0x7f9d900a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009830 tx=0x7f9d9c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:23.813+0000 7f9da5600640 1 -- 172.21.15.46:0/2698313327 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9da009d300 con 0x7f9da8101940 2024-04-17T07:55:23.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:23 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1093063836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:23.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:23 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1936960947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:23.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:23 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:23 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:23.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:23 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:23.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:23 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:55:23.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:23 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.021+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9da8062b60 con 0x7f9da8101940 2024-04-17T07:55:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.492+0000 7f9da5600640 1 -- 172.21.15.46:0/2698313327 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f9da009cc00 con 0x7f9da8101940 2024-04-17T07:55:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-17T07:55:24.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.498+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d900a3c30 msgr2=0x7f9d900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:24.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.498+0000 7f9dae400640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d900a3c30 0x7f9d900a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009830 tx=0x7f9d9c02e040 comp rx=0 tx=0).stop 2024-04-17T07:55:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.498+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8101940 msgr2=0x7f9da81a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.498+0000 7f9dae400640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8101940 0x7f9da81a1860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9da000eb10 tx=0x7f9da000ef10 comp rx=0 tx=0).stop 2024-04-17T07:55:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.499+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 shutdown_connections 2024-04-17T07:55:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.499+0000 7f9dae400640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d900a3c30 0x7f9d900a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.499+0000 7f9dae400640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9da8101940 0x7f9da81a1860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.499+0000 7f9dae400640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9da8100fa0 0x7f9da819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.499+0000 7f9dae400640 1 --2- 172.21.15.46:0/2698313327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9da8100640 0x7f9da819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.499+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 >> 172.21.15.46:0/2698313327 conn(0x7f9da80fc240 msgr2=0x7f9da80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.499+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 shutdown_connections 2024-04-17T07:55:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.500+0000 7f9dae400640 1 -- 172.21.15.46:0/2698313327 wait complete. 2024-04-17T07:55:24.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 -- 172.21.15.46:0/4188475322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88100e60 msgr2=0x7faf88101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 --2- 172.21.15.46:0/4188475322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88100e60 0x7faf88101280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf78009900 tx=0x7faf7802bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 -- 172.21.15.46:0/4188475322 shutdown_connections 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 --2- 172.21.15.46:0/4188475322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf88102a00 0x7faf88109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 --2- 172.21.15.46:0/4188475322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88102040 0x7faf881024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 --2- 172.21.15.46:0/4188475322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88100e60 0x7faf88101280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 -- 172.21.15.46:0/4188475322 >> 172.21.15.46:0/4188475322 conn(0x7faf880fc440 msgr2=0x7faf880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 -- 172.21.15.46:0/4188475322 shutdown_connections 2024-04-17T07:55:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.713+0000 7faf8cc00640 1 -- 172.21.15.46:0/4188475322 wait complete. 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.714+0000 7faf8cc00640 1 Processor -- start 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.714+0000 7faf8cc00640 1 -- start start 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf8cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88100e60 0x7faf8819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf8cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88102040 0x7faf8819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf8cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf88102a00 0x7faf881a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf8cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf8810cb20 con 0x7faf88102a00 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf8cc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf8810c9a0 con 0x7faf88100e60 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf8cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf8810cca0 con 0x7faf88102040 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88100e60 0x7faf8819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88100e60 0x7faf8819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40478/0 (socket says 172.21.15.46:40478) 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 -- 172.21.15.46:0/206309601 learned_addr learned my addr 172.21.15.46:0/206309601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf86800640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf88102a00 0x7faf881a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85400640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88102040 0x7faf8819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 -- 172.21.15.46:0/206309601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88102040 msgr2=0x7faf8819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88102040 0x7faf8819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 -- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf88102a00 msgr2=0x7faf881a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf88102a00 0x7faf881a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.715+0000 7faf85e00640 1 -- 172.21.15.46:0/206309601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf881a2140 con 0x7faf88100e60 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.716+0000 7faf85e00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88100e60 0x7faf8819a570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf78000c00 tx=0x7faf78005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.716+0000 7faf7ea00640 1 -- 172.21.15.46:0/206309601 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78041070 con 0x7faf88100e60 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.716+0000 7faf7ea00640 1 -- 172.21.15.46:0/206309601 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf780044d0 con 0x7faf88100e60 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.716+0000 7faf7ea00640 1 -- 172.21.15.46:0/206309601 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78034440 con 0x7faf88100e60 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.716+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf881a23d0 con 0x7faf88100e60 2024-04-17T07:55:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.716+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf881a28c0 con 0x7faf88100e60 2024-04-17T07:55:24.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.717+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf881a2b20 con 0x7faf88100e60 2024-04-17T07:55:24.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.719+0000 7faf7ea00640 1 -- 172.21.15.46:0/206309601 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf78044070 con 0x7faf88100e60 2024-04-17T07:55:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.720+0000 7faf7ea00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf6c0a3bb0 0x7faf6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.721+0000 7faf7ea00640 1 -- 172.21.15.46:0/206309601 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7faf780f1c50 con 0x7faf88100e60 2024-04-17T07:55:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.721+0000 7faf85400640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf6c0a3bb0 0x7faf6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.722+0000 7faf85400640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf6c0a3bb0 0x7faf6c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faf8819ba30 tx=0x7faf74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.725+0000 7faf7ea00640 1 -- 172.21.15.46:0/206309601 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faf780be270 con 0x7faf88100e60 2024-04-17T07:55:24.752 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:24 smithi062 ceph-mon[36507]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:24.752 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:24 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3966826025' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T07:55:24.752 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:55:24.753 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:24 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2698313327' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:55:24.753 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:24 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:55:24.821 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:24 smithi053 ceph-mon[36437]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:24.821 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:24 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3966826025' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T07:55:24.822 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:55:24.822 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:24 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2698313327' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:55:24.822 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:24 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:55:24.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:24 smithi046 ceph-mon[32038]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:24.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:24 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3966826025' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T07:55:24.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:55:24.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:24 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2698313327' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:55:24.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:24 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:55:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:24.929+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7faf881a3240 con 0x7faf88100e60 2024-04-17T07:55:25.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.495+0000 7faf7ea00640 1 -- 172.21.15.46:0/206309601 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7faf780c2570 con 0x7faf88100e60 2024-04-17T07:55:25.497 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-17T07:55:25.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.504+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf6c0a3bb0 msgr2=0x7faf6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:25.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.504+0000 7faf8cc00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf6c0a3bb0 0x7faf6c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faf8819ba30 tx=0x7faf74005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:25.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.504+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88100e60 msgr2=0x7faf8819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:25.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.504+0000 7faf8cc00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88100e60 0x7faf8819a570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf78000c00 tx=0x7faf78005d50 comp rx=0 tx=0).stop 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 shutdown_connections 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf6c0a3bb0 0x7faf6c0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf88102a00 0x7faf881a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf88102040 0x7faf8819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 --2- 172.21.15.46:0/206309601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf88100e60 0x7faf8819a570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 >> 172.21.15.46:0/206309601 conn(0x7faf880fc440 msgr2=0x7faf8806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 shutdown_connections 2024-04-17T07:55:25.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.505+0000 7faf8cc00640 1 -- 172.21.15.46:0/206309601 wait complete. 2024-04-17T07:55:25.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T07:55:25.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.745+0000 7feb35400640 1 -- 172.21.15.46:0/1141897999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb30105d70 msgr2=0x7feb3010c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.745+0000 7feb35400640 1 --2- 172.21.15.46:0/1141897999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb30105d70 0x7feb3010c600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb28009900 tx=0x7feb2802bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.746+0000 7feb35400640 1 -- 172.21.15.46:0/1141897999 shutdown_connections 2024-04-17T07:55:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.746+0000 7feb35400640 1 --2- 172.21.15.46:0/1141897999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb30105d70 0x7feb3010c600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.746+0000 7feb35400640 1 --2- 172.21.15.46:0/1141897999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb301053b0 0x7feb30105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.746+0000 7feb35400640 1 --2- 172.21.15.46:0/1141897999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb301041c0 0x7feb301045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.746+0000 7feb35400640 1 -- 172.21.15.46:0/1141897999 >> 172.21.15.46:0/1141897999 conn(0x7feb300ff750 msgr2=0x7feb30101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.746+0000 7feb35400640 1 -- 172.21.15.46:0/1141897999 shutdown_connections 2024-04-17T07:55:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.746+0000 7feb35400640 1 -- 172.21.15.46:0/1141897999 wait complete. 2024-04-17T07:55:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 Processor -- start 2024-04-17T07:55:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 -- start start 2024-04-17T07:55:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb301041c0 0x7feb3019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb301053b0 0x7feb3019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb30105d70 0x7feb301a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb3010fea0 con 0x7feb301053b0 2024-04-17T07:55:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb3010fd20 con 0x7feb301041c0 2024-04-17T07:55:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb35400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb30110020 con 0x7feb30105d70 2024-04-17T07:55:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb2e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb301041c0 0x7feb3019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb2f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb30105d70 0x7feb301a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2e800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb301041c0 0x7feb3019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40496/0 (socket says 172.21.15.46:40496) 2024-04-17T07:55:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2e800640 1 -- 172.21.15.46:0/1487461880 learned_addr learned my addr 172.21.15.46:0/1487461880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.747+0000 7feb27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb301053b0 0x7feb3019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2f200640 1 -- 172.21.15.46:0/1487461880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb301041c0 msgr2=0x7feb3019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2f200640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb301041c0 0x7feb3019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2f200640 1 -- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb301053b0 msgr2=0x7feb3019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2f200640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb301053b0 0x7feb3019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2f200640 1 -- 172.21.15.46:0/1487461880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb301a2080 con 0x7feb30105d70 2024-04-17T07:55:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.748+0000 7feb2f200640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb30105d70 0x7feb301a1a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb2802c170 tx=0x7feb2802c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.749+0000 7feb27400640 1 -- 172.21.15.46:0/1487461880 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28043020 con 0x7feb30105d70 2024-04-17T07:55:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.749+0000 7feb35400640 1 -- 172.21.15.46:0/1487461880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301a2310 con 0x7feb30105d70 2024-04-17T07:55:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.749+0000 7feb35400640 1 -- 172.21.15.46:0/1487461880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb301a2880 con 0x7feb30105d70 2024-04-17T07:55:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.749+0000 7feb27400640 1 -- 172.21.15.46:0/1487461880 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb280054e0 con 0x7feb30105d70 2024-04-17T07:55:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.749+0000 7feb27400640 1 -- 172.21.15.46:0/1487461880 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28003b60 con 0x7feb30105d70 2024-04-17T07:55:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.750+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb30062b60 con 0x7feb30105d70 2024-04-17T07:55:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.752+0000 7feb27400640 1 -- 172.21.15.46:0/1487461880 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb28046020 con 0x7feb30105d70 2024-04-17T07:55:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.754+0000 7feb27400640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb140a3bb0 0x7feb140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.754+0000 7feb27400640 1 -- 172.21.15.46:0/1487461880 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7feb280f1c40 con 0x7feb30105d70 2024-04-17T07:55:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.754+0000 7feb2e800640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb140a3bb0 0x7feb140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.755+0000 7feb2e800640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb140a3bb0 0x7feb140a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb20011440 tx=0x7feb20009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:25.757+0000 7feb27400640 1 -- 172.21.15.46:0/1487461880 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7feb2804d030 con 0x7feb30105d70 2024-04-17T07:55:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:25 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T07:55:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:25 smithi062 ceph-mon[36507]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T07:55:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:25 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/206309601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:55:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:25 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:55:25.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:25.804 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:25.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:25 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T07:55:25.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:25 smithi046 ceph-mon[32038]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T07:55:25.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:25 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/206309601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:55:25.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:25 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:55:25.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:25.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:25 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T07:55:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:25 smithi053 ceph-mon[36437]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T07:55:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:25 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/206309601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:55:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:25 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:55:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:25.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.014+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7feb30062ff0 con 0x7feb30105d70 2024-04-17T07:55:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.544+0000 7feb27400640 1 -- 172.21.15.46:0/1487461880 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7feb280be7c0 con 0x7feb30105d70 2024-04-17T07:55:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-04-17T07:55:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.548+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb140a3bb0 msgr2=0x7feb140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.548+0000 7feb24c00640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb140a3bb0 0x7feb140a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb20011440 tx=0x7feb20009500 comp rx=0 tx=0).stop 2024-04-17T07:55:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.548+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb30105d70 msgr2=0x7feb301a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.548+0000 7feb24c00640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb30105d70 0x7feb301a1a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb2802c170 tx=0x7feb2802c640 comp rx=0 tx=0).stop 2024-04-17T07:55:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.550+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 shutdown_connections 2024-04-17T07:55:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.550+0000 7feb24c00640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7feb140a3bb0 0x7feb140a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.550+0000 7feb24c00640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb30105d70 0x7feb301a1a10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.550+0000 7feb24c00640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb301053b0 0x7feb3019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.550+0000 7feb24c00640 1 --2- 172.21.15.46:0/1487461880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb301041c0 0x7feb3019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.550+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 >> 172.21.15.46:0/1487461880 conn(0x7feb300ff750 msgr2=0x7feb30108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.551+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 shutdown_connections 2024-04-17T07:55:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.551+0000 7feb24c00640 1 -- 172.21.15.46:0/1487461880 wait complete. 2024-04-17T07:55:26.574 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-17T07:55:26.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 -- 172.21.15.46:0/1627340108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458101980 msgr2=0x7f5458102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 --2- 172.21.15.46:0/1627340108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458101980 0x7f5458102200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5450009ca0 tx=0x7f545002c100 comp rx=0 tx=0).stop 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 -- 172.21.15.46:0/1627340108 shutdown_connections 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 --2- 172.21.15.46:0/1627340108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458101980 0x7f5458102200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 --2- 172.21.15.46:0/1627340108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458100fc0 0x7f5458101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 --2- 172.21.15.46:0/1627340108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5458100260 0x7f5458100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 -- 172.21.15.46:0/1627340108 >> 172.21.15.46:0/1627340108 conn(0x7f54580fbbb0 msgr2=0x7f54580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 -- 172.21.15.46:0/1627340108 shutdown_connections 2024-04-17T07:55:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.796+0000 7f545e400640 1 -- 172.21.15.46:0/1627340108 wait complete. 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.797+0000 7f545e400640 1 Processor -- start 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f545e400640 1 -- start start 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f545e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458100260 0x7f545819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f545e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5458100fc0 0x7f545819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f545e400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458101980 0x7f54581a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f545e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f545810fe70 con 0x7f5458100260 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f545e400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f545810fcf0 con 0x7f5458101980 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f545e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f545810fff0 con 0x7f5458100fc0 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f5457400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458100260 0x7f545819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f5457400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458100260 0x7f545819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37434/0 (socket says 172.21.15.46:37434) 2024-04-17T07:55:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f5457400640 1 -- 172.21.15.46:0/1725226046 learned_addr learned my addr 172.21.15.46:0/1725226046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f5457e00640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458101980 0x7f54581a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.798+0000 7f544fe00640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5458100fc0 0x7f545819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5457400640 1 -- 172.21.15.46:0/1725226046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5458100fc0 msgr2=0x7f545819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5457400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5458100fc0 0x7f545819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5457400640 1 -- 172.21.15.46:0/1725226046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458101980 msgr2=0x7f54581a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5457400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458101980 0x7f54581a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5457400640 1 -- 172.21.15.46:0/1725226046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54581a2070 con 0x7f5458100260 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5457400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458100260 0x7f545819a450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f544800e9f0 tx=0x7f544800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5454c00640 1 -- 172.21.15.46:0/1725226046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544800cd80 con 0x7f5458100260 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5454c00640 1 -- 172.21.15.46:0/1725226046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5448010910 con 0x7f5458100260 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f5454c00640 1 -- 172.21.15.46:0/1725226046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448018990 con 0x7f5458100260 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581a2360 con 0x7f5458100260 2024-04-17T07:55:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.799+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581a2900 con 0x7f5458100260 2024-04-17T07:55:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.800+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545810bd70 con 0x7f5458100260 2024-04-17T07:55:26.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.803+0000 7f5454c00640 1 -- 172.21.15.46:0/1725226046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5448010ac0 con 0x7f5458100260 2024-04-17T07:55:26.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-17T07:55:26.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: pgmap v315: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1487461880' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:26 smithi062 ceph-mon[36507]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:55:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.804+0000 7f5454c00640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f543c0a3c30 0x7f543c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.804+0000 7f5454c00640 1 -- 172.21.15.46:0/1725226046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f54480d0a70 con 0x7f5458100260 2024-04-17T07:55:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.804+0000 7f544fe00640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f543c0a3c30 0x7f543c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.805+0000 7f544fe00640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f543c0a3c30 0x7f543c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f545819b960 tx=0x7f5440005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:26.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:26.807+0000 7f5454c00640 1 -- 172.21.15.46:0/1725226046 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f544809d500 con 0x7f5458100260 2024-04-17T07:55:26.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-17T07:55:26.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T07:55:26.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: pgmap v315: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:26.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1487461880' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:26 smithi046 ceph-mon[32038]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:55:26.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-17T07:55:26.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T07:55:26.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: pgmap v315: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1487461880' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:55:26.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:26 smithi053 ceph-mon[36437]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:55:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.025+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f545802d5d0 con 0x7f5458100260 2024-04-17T07:55:27.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.329+0000 7f5454c00640 1 -- 172.21.15.46:0/1725226046 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f544809ce00 con 0x7f5458100260 2024-04-17T07:55:27.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f543c0a3c30 msgr2=0x7f543c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:27.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f543c0a3c30 0x7f543c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f545819b960 tx=0x7f5440005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458100260 msgr2=0x7f545819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458100260 0x7f545819a450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f544800e9f0 tx=0x7f544800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 shutdown_connections 2024-04-17T07:55:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f543c0a3c30 0x7f543c0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458101980 0x7f54581a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5458100fc0 0x7f545819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 --2- 172.21.15.46:0/1725226046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5458100260 0x7f545819a450 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.333+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 >> 172.21.15.46:0/1725226046 conn(0x7f54580fbbb0 msgr2=0x7f54580fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.334+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 shutdown_connections 2024-04-17T07:55:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.334+0000 7f545e400640 1 -- 172.21.15.46:0/1725226046 wait complete. 2024-04-17T07:55:27.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 -- 172.21.15.46:0/3151003304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0078500 msgr2=0x7faff0078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 --2- 172.21.15.46:0/3151003304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0078500 0x7faff0078da0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009ca0 tx=0x7fafe802c100 comp rx=0 tx=0).stop 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 -- 172.21.15.46:0/3151003304 shutdown_connections 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 --2- 172.21.15.46:0/3151003304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0078500 0x7faff0078da0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 --2- 172.21.15.46:0/3151003304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff0077b60 0x7faff0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 --2- 172.21.15.46:0/3151003304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0076920 0x7faff0076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 -- 172.21.15.46:0/3151003304 >> 172.21.15.46:0/3151003304 conn(0x7faff00ff630 msgr2=0x7faff0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 -- 172.21.15.46:0/3151003304 shutdown_connections 2024-04-17T07:55:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.588+0000 7faff6200640 1 -- 172.21.15.46:0/3151003304 wait complete. 2024-04-17T07:55:27.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.589+0000 7faff6200640 1 Processor -- start 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.589+0000 7faff6200640 1 -- start start 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff6200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0076920 0x7faff019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff0077b60 0x7faff019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0078500 0x7faff01a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faff0114130 con 0x7faff0078500 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff6200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faff0113fb0 con 0x7faff0076920 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faff01142b0 con 0x7faff0077b60 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0078500 0x7faff01a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0078500 0x7faff01a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37458/0 (socket says 172.21.15.46:37458) 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff4e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0076920 0x7faff019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7faff5800640 1 -- 172.21.15.46:0/3196009746 learned_addr learned my addr 172.21.15.46:0/3196009746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.590+0000 7fafe7e00640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff0077b60 0x7faff019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff4e00640 1 -- 172.21.15.46:0/3196009746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff0077b60 msgr2=0x7faff019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff4e00640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff0077b60 0x7faff019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff4e00640 1 -- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0078500 msgr2=0x7faff01a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff4e00640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0078500 0x7faff01a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff4e00640 1 -- 172.21.15.46:0/3196009746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff01a6350 con 0x7faff0076920 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff5800640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0078500 0x7faff01a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff4e00640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0076920 0x7faff019e780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e9f0 tx=0x7fafe000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7fafee000640 1 -- 172.21.15.46:0/3196009746 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000cd80 con 0x7faff0076920 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7fafee000640 1 -- 172.21.15.46:0/3196009746 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe00035f0 con 0x7faff0076920 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff01a6640 con 0x7faff0076920 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7fafee000640 1 -- 172.21.15.46:0/3196009746 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0010690 con 0x7faff0076920 2024-04-17T07:55:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.591+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a6b90 con 0x7faff0076920 2024-04-17T07:55:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.592+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff0110a10 con 0x7faff0076920 2024-04-17T07:55:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.594+0000 7fafee000640 1 -- 172.21.15.46:0/3196009746 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fafe0003b30 con 0x7faff0076920 2024-04-17T07:55:27.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.596+0000 7fafee000640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fafd40a3ba0 0x7fafd40a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:27.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.596+0000 7fafee000640 1 -- 172.21.15.46:0/3196009746 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7fafe00d0740 con 0x7faff0076920 2024-04-17T07:55:27.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.596+0000 7fafe7e00640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fafd40a3ba0 0x7fafd40a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.597+0000 7fafe7e00640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fafd40a3ba0 0x7fafd40a6060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faff019fc40 tx=0x7fafdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:27.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.600+0000 7fafee000640 1 -- 172.21.15.46:0/3196009746 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fafe009d090 con 0x7faff0076920 2024-04-17T07:55:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:27 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T07:55:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:27 smithi046 ceph-mon[32038]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T07:55:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:27 smithi046 ceph-mon[32038]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T07:55:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:27 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1725226046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-17T07:55:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:27 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1725226046' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-17T07:55:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:27 smithi046 ceph-mon[32038]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T07:55:27.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:27 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T07:55:27.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:27 smithi062 ceph-mon[36507]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T07:55:27.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:27 smithi062 ceph-mon[36507]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T07:55:27.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:27 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1725226046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-17T07:55:27.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:27 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1725226046' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-17T07:55:27.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:27 smithi062 ceph-mon[36507]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T07:55:27.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:27.810+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faff002d5d0 con 0x7faff0076920 2024-04-17T07:55:27.819 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:27 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T07:55:27.819 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:27 smithi053 ceph-mon[36437]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T07:55:27.820 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:27 smithi053 ceph-mon[36437]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T07:55:27.820 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:27 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1725226046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-17T07:55:27.820 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:27 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1725226046' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-17T07:55:27.820 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:27 smithi053 ceph-mon[36437]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T07:55:28.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: pgmap v318: 97 pgs: 30 creating+peering, 34 active+clean, 33 unknown; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-17T07:55:28.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] up:active 2024-04-17T07:55:28.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:28.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:28.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3196009746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:55:28.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:55:28.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T07:55:28.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:28 smithi053 ceph-mon[36437]: Cluster is now healthy 2024-04-17T07:55:28.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: pgmap v318: 97 pgs: 30 creating+peering, 34 active+clean, 33 unknown; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-17T07:55:28.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] up:active 2024-04-17T07:55:28.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:28.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:28.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3196009746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:55:28.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:55:28.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T07:55:28.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:28 smithi046 ceph-mon[32038]: Cluster is now healthy 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: pgmap v318: 97 pgs: 30 creating+peering, 34 active+clean, 33 unknown; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] up:active 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3196009746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T07:55:29.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:28 smithi062 ceph-mon[36507]: Cluster is now healthy 2024-04-17T07:55:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.567+0000 7fafee000640 1 -- 172.21.15.46:0/3196009746 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7fafe009c990 con 0x7faff0076920 2024-04-17T07:55:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' created 2024-04-17T07:55:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.574+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fafd40a3ba0 msgr2=0x7fafd40a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.574+0000 7faff6200640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fafd40a3ba0 0x7fafd40a6060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faff019fc40 tx=0x7fafdc005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.574+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0076920 msgr2=0x7faff019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.574+0000 7faff6200640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0076920 0x7faff019e780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e9f0 tx=0x7fafe000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 shutdown_connections 2024-04-17T07:55:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fafd40a3ba0 0x7fafd40a6060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faff0078500 0x7faff01a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faff0077b60 0x7faff019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 --2- 172.21.15.46:0/3196009746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faff0076920 0x7faff019e780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 >> 172.21.15.46:0/3196009746 conn(0x7faff00ff630 msgr2=0x7faff0101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 shutdown_connections 2024-04-17T07:55:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.575+0000 7faff6200640 1 -- 172.21.15.46:0/3196009746 wait complete. 2024-04-17T07:55:29.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-17T07:55:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 -- 172.21.15.46:0/2286977982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff678103fc0 msgr2=0x7ff6781043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2286977982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff678103fc0 0x7ff6781043e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009860 tx=0x7ff66c02bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 -- 172.21.15.46:0/2286977982 shutdown_connections 2024-04-17T07:55:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2286977982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678105b70 0x7ff67810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2286977982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6781051b0 0x7ff678105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2286977982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff678103fc0 0x7ff6781043e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 -- 172.21.15.46:0/2286977982 >> 172.21.15.46:0/2286977982 conn(0x7ff6780ff570 msgr2=0x7ff678101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 -- 172.21.15.46:0/2286977982 shutdown_connections 2024-04-17T07:55:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.801+0000 7ff67ea00640 1 -- 172.21.15.46:0/2286977982 wait complete. 2024-04-17T07:55:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.802+0000 7ff67ea00640 1 Processor -- start 2024-04-17T07:55:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67ea00640 1 -- start start 2024-04-17T07:55:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678103fc0 0x7ff67819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6781051b0 0x7ff67819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff678105b70 0x7ff6781a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff67810fc50 con 0x7ff678105b70 2024-04-17T07:55:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff67810fad0 con 0x7ff678103fc0 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff67810fdd0 con 0x7ff6781051b0 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678103fc0 0x7ff67819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6781051b0 0x7ff67819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff678105b70 0x7ff6781a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678103fc0 0x7ff67819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51390/0 (socket says 172.21.15.46:51390) 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.803+0000 7ff67d600640 1 -- 172.21.15.46:0/2928270889 learned_addr learned my addr 172.21.15.46:0/2928270889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67d600640 1 -- 172.21.15.46:0/2928270889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6781051b0 msgr2=0x7ff67819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67d600640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6781051b0 0x7ff67819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67d600640 1 -- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff678105b70 msgr2=0x7ff6781a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67d600640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff678105b70 0x7ff6781a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67d600640 1 -- 172.21.15.46:0/2928270889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6781a1ec0 con 0x7ff678103fc0 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67cc00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6781051b0 0x7ff67819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67d600640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678103fc0 0x7ff67819a2a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0349e0 tx=0x7ff66c032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff676000640 1 -- 172.21.15.46:0/2928270889 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c041020 con 0x7ff678103fc0 2024-04-17T07:55:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff676000640 1 -- 172.21.15.46:0/2928270889 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66c02c9f0 con 0x7ff678103fc0 2024-04-17T07:55:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff676000640 1 -- 172.21.15.46:0/2928270889 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c039040 con 0x7ff678103fc0 2024-04-17T07:55:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6781a2150 con 0x7ff678103fc0 2024-04-17T07:55:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.804+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6781a2690 con 0x7ff678103fc0 2024-04-17T07:55:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.806+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67810b2e0 con 0x7ff678103fc0 2024-04-17T07:55:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.807+0000 7ff676000640 1 -- 172.21.15.46:0/2928270889 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff66c044020 con 0x7ff678103fc0 2024-04-17T07:55:29.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.809+0000 7ff676000640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff6600a3c20 0x7ff6600a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:29.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.809+0000 7ff676000640 1 -- 172.21.15.46:0/2928270889 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7ff66c0f1df0 con 0x7ff678103fc0 2024-04-17T07:55:29.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.809+0000 7ff67cc00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff6600a3c20 0x7ff6600a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:29.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.810+0000 7ff67cc00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff6600a3c20 0x7ff6600a60e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff67819b7b0 tx=0x7ff668009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:29.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:29.813+0000 7ff676000640 1 -- 172.21.15.46:0/2928270889 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff66c04b020 con 0x7ff678103fc0 2024-04-17T07:55:29.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:29 smithi046 ceph-mon[32038]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T07:55:29.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:29 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T07:55:29.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:29 smithi046 ceph-mon[32038]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T07:55:29.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:29 smithi053 ceph-mon[36437]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T07:55:29.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:29 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T07:55:29.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:29 smithi053 ceph-mon[36437]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T07:55:30.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.022+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ff67802d5d0 con 0x7ff678103fc0 2024-04-17T07:55:30.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:29 smithi062 ceph-mon[36507]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T07:55:30.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:29 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T07:55:30.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:29 smithi062 ceph-mon[36507]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T07:55:30.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.570+0000 7ff676000640 1 -- 172.21.15.46:0/2928270889 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7ff66c0be550 con 0x7ff678103fc0 2024-04-17T07:55:30.572 INFO:teuthology.orchestra.run.smithi046.stderr:set pool 4 allow_ec_overwrites to true 2024-04-17T07:55:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.579+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff6600a3c20 msgr2=0x7ff6600a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.579+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff6600a3c20 0x7ff6600a60e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff67819b7b0 tx=0x7ff668009260 comp rx=0 tx=0).stop 2024-04-17T07:55:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.579+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678103fc0 msgr2=0x7ff67819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.579+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678103fc0 0x7ff67819a2a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0349e0 tx=0x7ff66c032430 comp rx=0 tx=0).stop 2024-04-17T07:55:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.580+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 shutdown_connections 2024-04-17T07:55:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.580+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff6600a3c20 0x7ff6600a60e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.580+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff678105b70 0x7ff6781a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.580+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6781051b0 0x7ff67819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.580+0000 7ff67ea00640 1 --2- 172.21.15.46:0/2928270889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff678103fc0 0x7ff67819a2a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.580+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 >> 172.21.15.46:0/2928270889 conn(0x7ff6780ff570 msgr2=0x7ff6781084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.581+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 shutdown_connections 2024-04-17T07:55:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.581+0000 7ff67ea00640 1 -- 172.21.15.46:0/2928270889 wait complete. 2024-04-17T07:55:30.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-17T07:55:30.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.806+0000 7f6cdc600640 1 -- 172.21.15.46:0/3197516738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 msgr2=0x7f6cd4078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:30.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.806+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3197516738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 0x7f6cd4078da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009ca0 tx=0x7f6ccc02c100 comp rx=0 tx=0).stop 2024-04-17T07:55:30.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.807+0000 7f6cdc600640 1 -- 172.21.15.46:0/3197516738 shutdown_connections 2024-04-17T07:55:30.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.807+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3197516738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 0x7f6cd4078da0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.807+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3197516738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4077b60 0x7f6cd4077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.807+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3197516738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd4076920 0x7f6cd4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.807+0000 7f6cdc600640 1 -- 172.21.15.46:0/3197516738 >> 172.21.15.46:0/3197516738 conn(0x7f6cd40ff720 msgr2=0x7f6cd4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.807+0000 7f6cdc600640 1 -- 172.21.15.46:0/3197516738 shutdown_connections 2024-04-17T07:55:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.807+0000 7f6cdc600640 1 -- 172.21.15.46:0/3197516738 wait complete. 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.808+0000 7f6cdc600640 1 Processor -- start 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.808+0000 7f6cdc600640 1 -- start start 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cdc600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4076920 0x7f6cd419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cdc600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd4077b60 0x7f6cd419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cdc600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 0x7f6cd41a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cdc600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4114240 con 0x7f6cd4076920 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cdc600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41140c0 con 0x7f6cd4077b60 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cdc600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41143c0 con 0x7f6cd4078500 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4076920 0x7f6cd419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4076920 0x7f6cd419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51792/0 (socket says 172.21.15.46:51792) 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 -- 172.21.15.46:0/3760811653 learned_addr learned my addr 172.21.15.46:0/3760811653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9200640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd4077b60 0x7f6cd419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cda600640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 0x7f6cd41a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 -- 172.21.15.46:0/3760811653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 msgr2=0x7f6cd41a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 0x7f6cd41a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 -- 172.21.15.46:0/3760811653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd4077b60 msgr2=0x7f6cd419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd4077b60 0x7f6cd419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.809+0000 7f6cd9c00640 1 -- 172.21.15.46:0/3760811653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd41a6490 con 0x7f6cd4076920 2024-04-17T07:55:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.810+0000 7f6cd9c00640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4076920 0x7f6cd419e870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800e9f0 tx=0x7f6cc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.810+0000 7f6cd2000640 1 -- 172.21.15.46:0/3760811653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800cd80 con 0x7f6cd4076920 2024-04-17T07:55:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.810+0000 7f6cd2000640 1 -- 172.21.15.46:0/3760811653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8005500 con 0x7f6cd4076920 2024-04-17T07:55:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.810+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd41a6780 con 0x7f6cd4076920 2024-04-17T07:55:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.810+0000 7f6cd2000640 1 -- 172.21.15.46:0/3760811653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8010690 con 0x7f6cd4076920 2024-04-17T07:55:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.810+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41a6c80 con 0x7f6cd4076920 2024-04-17T07:55:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.811+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd4110b20 con 0x7f6cd4076920 2024-04-17T07:55:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.814+0000 7f6cd2000640 1 -- 172.21.15.46:0/3760811653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6cc80046c0 con 0x7f6cd4076920 2024-04-17T07:55:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.815+0000 7f6cd2000640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6cbc0a3bb0 0x7f6cbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.815+0000 7f6cd2000640 1 -- 172.21.15.46:0/3760811653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f6cc80d0c80 con 0x7f6cd4076920 2024-04-17T07:55:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.816+0000 7f6cd9200640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6cbc0a3bb0 0x7f6cbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.816+0000 7f6cd9200640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6cbc0a3bb0 0x7f6cbc0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00095f0 tx=0x7f6cc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:30.818+0000 7f6cd2000640 1 -- 172.21.15.46:0/3760811653 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6cc8053ff0 con 0x7f6cd4076920 2024-04-17T07:55:30.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:30 smithi046 ceph-mon[32038]: pgmap v320: 97 pgs: 30 creating+peering, 65 active+clean, 2 unknown; 578 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-04-17T07:55:30.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:30 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2928270889' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:55:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:30 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:55:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:30 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-17T07:55:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:30 smithi046 ceph-mon[32038]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T07:55:30.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:30 smithi053 ceph-mon[36437]: pgmap v320: 97 pgs: 30 creating+peering, 65 active+clean, 2 unknown; 578 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-04-17T07:55:30.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:30 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2928270889' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:55:30.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:30 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:55:30.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:30 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-17T07:55:30.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:30 smithi053 ceph-mon[36437]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T07:55:31.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:30 smithi062 ceph-mon[36507]: pgmap v320: 97 pgs: 30 creating+peering, 65 active+clean, 2 unknown; 578 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2024-04-17T07:55:31.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:30 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2928270889' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:55:31.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:30 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:55:31.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:30 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-17T07:55:31.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:30 smithi062 ceph-mon[36507]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T07:55:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.063+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f6cd402d5d0 con 0x7f6cd4076920 2024-04-17T07:55:31.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.615+0000 7f6cd2000640 1 -- 172.21.15.46:0/3760811653 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f6cc809d3e0 con 0x7f6cd4076920 2024-04-17T07:55:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:added data pool 4 to fsmap 2024-04-17T07:55:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6cbc0a3bb0 msgr2=0x7f6cbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6cbc0a3bb0 0x7f6cbc0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00095f0 tx=0x7f6cc0005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4076920 msgr2=0x7f6cd419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4076920 0x7f6cd419e870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800e9f0 tx=0x7f6cc800edf0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 shutdown_connections 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6cbc0a3bb0 0x7f6cbc0a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6cd4078500 0x7f6cd41a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd4077b60 0x7f6cd419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 --2- 172.21.15.46:0/3760811653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd4076920 0x7f6cd419e870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 >> 172.21.15.46:0/3760811653 conn(0x7f6cd40ff720 msgr2=0x7f6cd4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 shutdown_connections 2024-04-17T07:55:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.619+0000 7f6cdc600640 1 -- 172.21.15.46:0/3760811653 wait complete. 2024-04-17T07:55:31.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:31.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:31 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3760811653' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-17T07:55:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 -- 172.21.15.46:0/833141776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 msgr2=0x7f3880102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 --2- 172.21.15.46:0/833141776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880102280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3874009ca0 tx=0x7f387402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 -- 172.21.15.46:0/833141776 shutdown_connections 2024-04-17T07:55:31.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 --2- 172.21.15.46:0/833141776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3880103a00 0x7f388010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 --2- 172.21.15.46:0/833141776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3880103040 0x7f38801034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 --2- 172.21.15.46:0/833141776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880102280 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 -- 172.21.15.46:0/833141776 >> 172.21.15.46:0/833141776 conn(0x7f38800fd420 msgr2=0x7f38800ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 -- 172.21.15.46:0/833141776 shutdown_connections 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.870+0000 7f3886800640 1 -- 172.21.15.46:0/833141776 wait complete. 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.871+0000 7f3886800640 1 Processor -- start 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.871+0000 7f3886800640 1 -- start start 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3886800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3886800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3880103040 0x7f3880074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3886800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3880103a00 0x7f3880074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3886800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f388010db40 con 0x7f3880101e60 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3886800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f388010d9c0 con 0x7f3880103040 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3886800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f388010dcc0 con 0x7f3880103a00 2024-04-17T07:55:31.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51814/0 (socket says 172.21.15.46:51814) 2024-04-17T07:55:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 -- 172.21.15.46:0/2766333249 learned_addr learned my addr 172.21.15.46:0/2766333249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885e00640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3880103a00 0x7f3880074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3884a00640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3880103040 0x7f3880074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 -- 172.21.15.46:0/2766333249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3880103a00 msgr2=0x7f3880074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3880103a00 0x7f3880074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 -- 172.21.15.46:0/2766333249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3880103040 msgr2=0x7f3880074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3880103040 0x7f3880074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.872+0000 7f3885400640 1 -- 172.21.15.46:0/2766333249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3880075120 con 0x7f3880101e60 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.873+0000 7f3885400640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880075d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f38740324c0 tx=0x7f3874032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.873+0000 7f387e000640 1 -- 172.21.15.46:0/2766333249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874041020 con 0x7f3880101e60 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.873+0000 7f387e000640 1 -- 172.21.15.46:0/2766333249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874034cd0 con 0x7f3880101e60 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.873+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38801a2270 con 0x7f3880101e60 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.873+0000 7f387e000640 1 -- 172.21.15.46:0/2766333249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874039040 con 0x7f3880101e60 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.873+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38801a2810 con 0x7f3880101e60 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.874+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3880109200 con 0x7f3880101e60 2024-04-17T07:55:31.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.877+0000 7f387e000640 1 -- 172.21.15.46:0/2766333249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3874044020 con 0x7f3880101e60 2024-04-17T07:55:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.878+0000 7f387e000640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38680a3c20 0x7f38680a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.878+0000 7f387e000640 1 -- 172.21.15.46:0/2766333249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f38740f28c0 con 0x7f3880101e60 2024-04-17T07:55:31.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.878+0000 7f3884a00640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38680a3c20 0x7f38680a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.879+0000 7f3884a00640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38680a3c20 0x7f38680a60e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3880075a80 tx=0x7f386c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:31.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:31.881+0000 7f387e000640 1 -- 172.21.15.46:0/2766333249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f387404b020 con 0x7f3880101e60 2024-04-17T07:55:31.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:31 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3760811653' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-17T07:55:31.971 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: 2024-04-17T07:55:31.586+0000 7f2180e94700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T07:55:32.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:31 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3760811653' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-17T07:55:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.132+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f388002d5d0 con 0x7f3880101e60 2024-04-17T07:55:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.134+0000 7f387e000640 1 -- 172.21.15.46:0/2766333249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f38740bf090 con 0x7f3880101e60 2024-04-17T07:55:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:32.134 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:31.065069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-17T07:55:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38680a3c20 msgr2=0x7f38680a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38680a3c20 0x7f38680a60e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3880075a80 tx=0x7f386c005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 msgr2=0x7f3880075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880075d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f38740324c0 tx=0x7f3874032f40 comp rx=0 tx=0).stop 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 shutdown_connections 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f38680a3c20 0x7f38680a60e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3880103a00 0x7f3880074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3880103040 0x7f3880074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 --2- 172.21.15.46:0/2766333249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3880101e60 0x7f3880075d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 >> 172.21.15.46:0/2766333249 conn(0x7f38800fd420 msgr2=0x7f38801063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 shutdown_connections 2024-04-17T07:55:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.137+0000 7f3886800640 1 -- 172.21.15.46:0/2766333249 wait complete. 2024-04-17T07:55:32.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 -- 172.21.15.46:0/1930090962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c1039a0 msgr2=0x7f144c10a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 --2- 172.21.15.46:0/1930090962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c1039a0 0x7f144c10a2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1444009ca0 tx=0x7f144402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 -- 172.21.15.46:0/1930090962 shutdown_connections 2024-04-17T07:55:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 --2- 172.21.15.46:0/1930090962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c1039a0 0x7f144c10a2c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 --2- 172.21.15.46:0/1930090962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144c102fe0 0x7f144c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 --2- 172.21.15.46:0/1930090962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f144c101e90 0x7f144c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 -- 172.21.15.46:0/1930090962 >> 172.21.15.46:0/1930090962 conn(0x7f144c0fd420 msgr2=0x7f144c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 -- 172.21.15.46:0/1930090962 shutdown_connections 2024-04-17T07:55:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.380+0000 7f144be00640 1 -- 172.21.15.46:0/1930090962 wait complete. 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.381+0000 7f144be00640 1 Processor -- start 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144be00640 1 -- start start 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c101e90 0x7f144c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f144c102fe0 0x7f144c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144c1039a0 0x7f144c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10db20 con 0x7f144c101e90 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10d9a0 con 0x7f144c1039a0 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10dca0 con 0x7f144c102fe0 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c101e90 0x7f144c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c101e90 0x7f144c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51834/0 (socket says 172.21.15.46:51834) 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144aa00640 1 -- 172.21.15.46:0/4023636604 learned_addr learned my addr 172.21.15.46:0/4023636604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.382+0000 7f144b400640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144c1039a0 0x7f144c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f1442000640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f144c102fe0 0x7f144c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f144b400640 1 -- 172.21.15.46:0/4023636604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f144c102fe0 msgr2=0x7f144c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f144b400640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f144c102fe0 0x7f144c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f144b400640 1 -- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c101e90 msgr2=0x7f144c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f144b400640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c101e90 0x7f144c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f144b400640 1 -- 172.21.15.46:0/4023636604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c1a1ef0 con 0x7f144c1039a0 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f1442000640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f144c102fe0 0x7f144c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f144b400640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144c1039a0 0x7f144c1a1880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f144402c630 tx=0x7f1444032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f1443e00640 1 -- 172.21.15.46:0/4023636604 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444041020 con 0x7f144c1039a0 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f1443e00640 1 -- 172.21.15.46:0/4023636604 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1444034cd0 con 0x7f144c1039a0 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c1a2180 con 0x7f144c1039a0 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.383+0000 7f1443e00640 1 -- 172.21.15.46:0/4023636604 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444039040 con 0x7f144c1039a0 2024-04-17T07:55:32.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.384+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1a2640 con 0x7f144c1039a0 2024-04-17T07:55:32.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.385+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c1a28a0 con 0x7f144c1039a0 2024-04-17T07:55:32.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.387+0000 7f1443e00640 1 -- 172.21.15.46:0/4023636604 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1444044020 con 0x7f144c1039a0 2024-04-17T07:55:32.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.388+0000 7f1443e00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14300a3ba0 0x7f14300a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.388+0000 7f1443e00640 1 -- 172.21.15.46:0/4023636604 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f14440f2430 con 0x7f144c1039a0 2024-04-17T07:55:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.388+0000 7f144aa00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14300a3ba0 0x7f14300a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.389+0000 7f144aa00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14300a3ba0 0x7f14300a6060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f143c009c90 tx=0x7f143c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:32.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.392+0000 7f1443e00640 1 -- 172.21.15.46:0/4023636604 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f144404b020 con 0x7f144c1039a0 2024-04-17T07:55:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.605+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f144c1a2f60 con 0x7f144c1039a0 2024-04-17T07:55:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.612+0000 7f1443e00640 1 -- 172.21.15.46:0/4023636604 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f14440bec60 con 0x7f144c1039a0 2024-04-17T07:55:32.612 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:32.612 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14300a3ba0 msgr2=0x7f14300a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14300a3ba0 0x7f14300a6060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f143c009c90 tx=0x7f143c005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144c1039a0 msgr2=0x7f144c1a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144c1039a0 0x7f144c1a1880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f144402c630 tx=0x7f1444032f20 comp rx=0 tx=0).stop 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 shutdown_connections 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14300a3ba0 0x7f14300a6060 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144c1039a0 0x7f144c1a1880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f144c102fe0 0x7f144c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 --2- 172.21.15.46:0/4023636604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f144c101e90 0x7f144c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.615+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 >> 172.21.15.46:0/4023636604 conn(0x7f144c0fd420 msgr2=0x7f144c074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.616+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 shutdown_connections 2024-04-17T07:55:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.616+0000 7f144be00640 1 -- 172.21.15.46:0/4023636604 wait complete. 2024-04-17T07:55:32.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:32.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:32 smithi046 ceph-mon[32038]: pgmap v323: 129 pgs: 1 creating+activating, 4 creating+peering, 113 active+clean, 11 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s wr, 8 op/s 2024-04-17T07:55:32.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:32 smithi046 ceph-mon[32038]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:55:32.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:32 smithi046 ceph-mon[32038]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T07:55:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:32 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3760811653' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-17T07:55:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:32 smithi046 ceph-mon[32038]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:32.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:32 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2766333249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 -- 172.21.15.46:0/4125816513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8101f90 msgr2=0x7f27a8102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 --2- 172.21.15.46:0/4125816513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8101f90 0x7f27a8102410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2794009900 tx=0x7f279402bc50 comp rx=0 tx=0).stop 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 -- 172.21.15.46:0/4125816513 shutdown_connections 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 --2- 172.21.15.46:0/4125816513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 0x7f27a81091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 --2- 172.21.15.46:0/4125816513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8101f90 0x7f27a8102410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 --2- 172.21.15.46:0/4125816513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8100da0 0x7f27a81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 -- 172.21.15.46:0/4125816513 >> 172.21.15.46:0/4125816513 conn(0x7f27a80fc330 msgr2=0x7f27a80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 -- 172.21.15.46:0/4125816513 shutdown_connections 2024-04-17T07:55:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.861+0000 7f27acc00640 1 -- 172.21.15.46:0/4125816513 wait complete. 2024-04-17T07:55:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.862+0000 7f27acc00640 1 Processor -- start 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.862+0000 7f27acc00640 1 -- start start 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27acc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8100da0 0x7f27a810b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27acc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8101f90 0x7f27a8109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27acc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 0x7f27a810a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27acc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27a810dba0 con 0x7f27a8102950 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27acc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f27a810da20 con 0x7f27a8101f90 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27acc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27a810dd20 con 0x7f27a8100da0 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 0x7f27a810a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8100da0 0x7f27a810b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 0x7f27a810a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51856/0 (socket says 172.21.15.46:51856) 2024-04-17T07:55:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7e00640 1 -- 172.21.15.46:0/620351471 learned_addr learned my addr 172.21.15.46:0/620351471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8100da0 0x7f27a810b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39394/0 (socket says 172.21.15.46:39394) 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a6a00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8101f90 0x7f27a8109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7400640 1 -- 172.21.15.46:0/620351471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8101f90 msgr2=0x7f27a8109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7400640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8101f90 0x7f27a8109d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7400640 1 -- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 msgr2=0x7f27a810a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.863+0000 7f27a7400640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 0x7f27a810a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f27a7400640 1 -- 172.21.15.46:0/620351471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a810ab30 con 0x7f27a8100da0 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f27a7e00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 0x7f27a810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f27a7400640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8100da0 0x7f27a810b720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f279800e9c0 tx=0x7f279800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f279fe00640 1 -- 172.21.15.46:0/620351471 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279800ccc0 con 0x7f27a8100da0 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f279fe00640 1 -- 172.21.15.46:0/620351471 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27980035f0 con 0x7f27a8100da0 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a81a2300 con 0x7f27a8100da0 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f279fe00640 1 -- 172.21.15.46:0/620351471 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279800ce60 con 0x7f27a8100da0 2024-04-17T07:55:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.864+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a81a2870 con 0x7f27a8100da0 2024-04-17T07:55:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.865+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a81080c0 con 0x7f27a8100da0 2024-04-17T07:55:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.867+0000 7f279fe00640 1 -- 172.21.15.46:0/620351471 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f27980037a0 con 0x7f27a8100da0 2024-04-17T07:55:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.869+0000 7f279fe00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f278c0a3c20 0x7f278c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.869+0000 7f279fe00640 1 -- 172.21.15.46:0/620351471 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f27980d09e0 con 0x7f27a8100da0 2024-04-17T07:55:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.869+0000 7f27a6a00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f278c0a3c20 0x7f278c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.870+0000 7f27a6a00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f278c0a3c20 0x7f278c0a60e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f279402c260 tx=0x7f2794000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:32.874+0000 7f279fe00640 1 -- 172.21.15.46:0/620351471 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f279809d120 con 0x7f27a8100da0 2024-04-17T07:55:32.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:32 smithi053 ceph-mon[36437]: pgmap v323: 129 pgs: 1 creating+activating, 4 creating+peering, 113 active+clean, 11 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s wr, 8 op/s 2024-04-17T07:55:32.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:32 smithi053 ceph-mon[36437]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:55:32.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:32 smithi053 ceph-mon[36437]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T07:55:32.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:32 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3760811653' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-17T07:55:32.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:32 smithi053 ceph-mon[36437]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:32.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:32 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2766333249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:33.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:32 smithi062 ceph-mon[36507]: pgmap v323: 129 pgs: 1 creating+activating, 4 creating+peering, 113 active+clean, 11 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s wr, 8 op/s 2024-04-17T07:55:33.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:32 smithi062 ceph-mon[36507]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:55:33.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:32 smithi062 ceph-mon[36507]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T07:55:33.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:32 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3760811653' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-17T07:55:33.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:32 smithi062 ceph-mon[36507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:33.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:32 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2766333249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.081+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27a8062b60 con 0x7f27a8100da0 2024-04-17T07:55:33.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.084+0000 7f279fe00640 1 -- 172.21.15.46:0/620351471 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f279809ca20 con 0x7f27a8100da0 2024-04-17T07:55:33.084 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:33.084 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f278c0a3c20 msgr2=0x7f278c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f278c0a3c20 0x7f278c0a60e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f279402c260 tx=0x7f2794000bc0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8100da0 msgr2=0x7f27a810b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8100da0 0x7f27a810b720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f279800e9c0 tx=0x7f279800ee80 comp rx=0 tx=0).stop 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 shutdown_connections 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f278c0a3c20 0x7f278c0a60e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a8102950 0x7f27a810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8101f90 0x7f27a8109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 --2- 172.21.15.46:0/620351471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a8100da0 0x7f27a810b720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 >> 172.21.15.46:0/620351471 conn(0x7f27a80fc330 msgr2=0x7f27a8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 shutdown_connections 2024-04-17T07:55:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.087+0000 7f27acc00640 1 -- 172.21.15.46:0/620351471 wait complete. 2024-04-17T07:55:33.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 -- 172.21.15.46:0/2419366905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4077b60 msgr2=0x7f48e4077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2419366905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4077b60 0x7f48e4077fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009900 tx=0x7f48cc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 -- 172.21.15.46:0/2419366905 shutdown_connections 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2419366905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48e4078500 0x7f48e4078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2419366905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4077b60 0x7f48e4077fc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2419366905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4076920 0x7f48e4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 -- 172.21.15.46:0/2419366905 >> 172.21.15.46:0/2419366905 conn(0x7f48e40ff630 msgr2=0x7f48e4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 -- 172.21.15.46:0/2419366905 shutdown_connections 2024-04-17T07:55:33.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.357+0000 7f48e3e00640 1 -- 172.21.15.46:0/2419366905 wait complete. 2024-04-17T07:55:33.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.358+0000 7f48e3e00640 1 Processor -- start 2024-04-17T07:55:33.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3e00640 1 -- start start 2024-04-17T07:55:33.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4076920 0x7f48e419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4077b60 0x7f48e419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48e4078500 0x7f48e41a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4114170 con 0x7f48e4078500 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4113ff0 con 0x7f48e4077b60 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48e41142f0 con 0x7f48e4076920 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48da000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4077b60 0x7f48e419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48e4078500 0x7f48e41a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4076920 0x7f48e419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48da000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4077b60 0x7f48e419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51468/0 (socket says 172.21.15.46:51468) 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.359+0000 7f48da000640 1 -- 172.21.15.46:0/2573634173 learned_addr learned my addr 172.21.15.46:0/2573634173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.360+0000 7f48e2a00640 1 -- 172.21.15.46:0/2573634173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4077b60 msgr2=0x7f48e419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.360+0000 7f48e2a00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4077b60 0x7f48e419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.360+0000 7f48e2a00640 1 -- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48e4078500 msgr2=0x7f48e41a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.360+0000 7f48e2a00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48e4078500 0x7f48e41a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.360+0000 7f48e2a00640 1 -- 172.21.15.46:0/2573634173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e41a6410 con 0x7f48e4076920 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.360+0000 7f48da000640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4077b60 0x7f48e419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.360+0000 7f48e2a00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4076920 0x7f48e419e7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e9f0 tx=0x7f48d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.361+0000 7f48dbe00640 1 -- 172.21.15.46:0/2573634173 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d400ccc0 con 0x7f48e4076920 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.361+0000 7f48dbe00640 1 -- 172.21.15.46:0/2573634173 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d4005020 con 0x7f48e4076920 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.361+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e41a6700 con 0x7f48e4076920 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.361+0000 7f48dbe00640 1 -- 172.21.15.46:0/2573634173 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d40049e0 con 0x7f48e4076920 2024-04-17T07:55:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.361+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41a6bd0 con 0x7f48e4076920 2024-04-17T07:55:33.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.362+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e4110a10 con 0x7f48e4076920 2024-04-17T07:55:33.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.364+0000 7f48dbe00640 1 -- 172.21.15.46:0/2573634173 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48d4005560 con 0x7f48e4076920 2024-04-17T07:55:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.365+0000 7f48dbe00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48c80a3ba0 0x7f48c80a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.366+0000 7f48dbe00640 1 -- 172.21.15.46:0/2573634173 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f48d40d07b0 con 0x7f48e4076920 2024-04-17T07:55:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.366+0000 7f48da000640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48c80a3ba0 0x7f48c80a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.366+0000 7f48da000640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48c80a3ba0 0x7f48c80a6060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f48cc02c8e0 tx=0x7f48cc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.368+0000 7f48dbe00640 1 -- 172.21.15.46:0/2573634173 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f48d40051d0 con 0x7f48e4076920 2024-04-17T07:55:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.619+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48e402d5d0 con 0x7f48e4076920 2024-04-17T07:55:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.620+0000 7f48dbe00640 1 -- 172.21.15.46:0/2573634173 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f48d409cef0 con 0x7f48e4076920 2024-04-17T07:55:33.619 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:33.620 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:31.065069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-17T07:55:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-04-17T07:55:33.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48c80a3ba0 msgr2=0x7f48c80a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48c80a3ba0 0x7f48c80a6060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f48cc02c8e0 tx=0x7f48cc02df90 comp rx=0 tx=0).stop 2024-04-17T07:55:33.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4076920 msgr2=0x7f48e419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4076920 0x7f48e419e7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e9f0 tx=0x7f48d400eeb0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 shutdown_connections 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48c80a3ba0 0x7f48c80a6060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48e4078500 0x7f48e41a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48e4077b60 0x7f48e419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 --2- 172.21.15.46:0/2573634173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48e4076920 0x7f48e419e7f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 >> 172.21.15.46:0/2573634173 conn(0x7f48e40ff630 msgr2=0x7f48e4101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 shutdown_connections 2024-04-17T07:55:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.623+0000 7f48e3e00640 1 -- 172.21.15.46:0/2573634173 wait complete. 2024-04-17T07:55:33.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.852+0000 7fef6fe00640 1 -- 172.21.15.46:0/839387880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70077b60 msgr2=0x7fef70077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.852+0000 7fef6fe00640 1 --2- 172.21.15.46:0/839387880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70077b60 0x7fef70077fc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009900 tx=0x7fef5c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:55:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.853+0000 7fef6fe00640 1 -- 172.21.15.46:0/839387880 shutdown_connections 2024-04-17T07:55:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.853+0000 7fef6fe00640 1 --2- 172.21.15.46:0/839387880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70078500 0x7fef70078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.853+0000 7fef6fe00640 1 --2- 172.21.15.46:0/839387880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70077b60 0x7fef70077fc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.853+0000 7fef6fe00640 1 --2- 172.21.15.46:0/839387880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef70076920 0x7fef70076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.853+0000 7fef6fe00640 1 -- 172.21.15.46:0/839387880 >> 172.21.15.46:0/839387880 conn(0x7fef700ff630 msgr2=0x7fef70101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.853+0000 7fef6fe00640 1 -- 172.21.15.46:0/839387880 shutdown_connections 2024-04-17T07:55:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.853+0000 7fef6fe00640 1 -- 172.21.15.46:0/839387880 wait complete. 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 Processor -- start 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 -- start start 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70076920 0x7fef7019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef70077b60 0x7fef7019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70078500 0x7fef701a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef701140f0 con 0x7fef70078500 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef70113f70 con 0x7fef70077b60 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fef70114270 con 0x7fef70076920 2024-04-17T07:55:33.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70076920 0x7fef7019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70076920 0x7fef7019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39440/0 (socket says 172.21.15.46:39440) 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.854+0000 7fef6ea00640 1 -- 172.21.15.46:0/2153486396 learned_addr learned my addr 172.21.15.46:0/2153486396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6f400640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70078500 0x7fef701a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6e000640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef70077b60 0x7fef7019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6ea00640 1 -- 172.21.15.46:0/2153486396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef70077b60 msgr2=0x7fef7019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6ea00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef70077b60 0x7fef7019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6ea00640 1 -- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70078500 msgr2=0x7fef701a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6ea00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70078500 0x7fef701a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6ea00640 1 -- 172.21.15.46:0/2153486396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef701a64a0 con 0x7fef70076920 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6f400640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70078500 0x7fef701a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6ea00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70076920 0x7fef7019e880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef6000ea60 tx=0x7fef6000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef67400640 1 -- 172.21.15.46:0/2153486396 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6000cde0 con 0x7fef70076920 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef67400640 1 -- 172.21.15.46:0/2153486396 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef600035f0 con 0x7fef70076920 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef701a6790 con 0x7fef70076920 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.855+0000 7fef67400640 1 -- 172.21.15.46:0/2153486396 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60010670 con 0x7fef70076920 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.856+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef701a6d00 con 0x7fef70076920 2024-04-17T07:55:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.857+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef70110a10 con 0x7fef70076920 2024-04-17T07:55:33.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.858+0000 7fef67400640 1 -- 172.21.15.46:0/2153486396 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef600037a0 con 0x7fef70076920 2024-04-17T07:55:33.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:33 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4023636604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:33.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:33 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/620351471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.860+0000 7fef67400640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fef540a3c20 0x7fef540a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.860+0000 7fef67400640 1 -- 172.21.15.46:0/2153486396 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fef600d0a40 con 0x7fef70076920 2024-04-17T07:55:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.860+0000 7fef6e000640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fef540a3c20 0x7fef540a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.861+0000 7fef6e000640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fef540a3c20 0x7fef540a60e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006010 tx=0x7fef5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:33.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:33.864+0000 7fef67400640 1 -- 172.21.15.46:0/2153486396 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fef6009d180 con 0x7fef70076920 2024-04-17T07:55:33.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:33 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4023636604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:33.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:33 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/620351471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:34.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:33 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4023636604' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:34.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:33 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/620351471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:34.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.072+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef70062b60 con 0x7fef70076920 2024-04-17T07:55:34.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.074+0000 7fef67400640 1 -- 172.21.15.46:0/2153486396 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7fef600bd0c0 con 0x7fef70076920 2024-04-17T07:55:34.074 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:34.074 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fef540a3c20 msgr2=0x7fef540a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fef540a3c20 0x7fef540a60e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006010 tx=0x7fef5c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70076920 msgr2=0x7fef7019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70076920 0x7fef7019e880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef6000ea60 tx=0x7fef6000ef20 comp rx=0 tx=0).stop 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 shutdown_connections 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fef540a3c20 0x7fef540a60e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef70078500 0x7fef701a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef70077b60 0x7fef7019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 --2- 172.21.15.46:0/2153486396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef70076920 0x7fef7019e880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 >> 172.21.15.46:0/2153486396 conn(0x7fef700ff630 msgr2=0x7fef70101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 shutdown_connections 2024-04-17T07:55:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.077+0000 7fef6fe00640 1 -- 172.21.15.46:0/2153486396 wait complete. 2024-04-17T07:55:34.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T07:55:26.016968+0000', 'modified': '2024-04-17T07:55:31.065069+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24596}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/262156089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 262156089}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 262156089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24596, 'qdb_cluster': [24596]} 2024-04-17T07:55:34.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T07:55:34.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 -- 172.21.15.46:0/4142482150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d41051d0 msgr2=0x7f36d4105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 --2- 172.21.15.46:0/4142482150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d41051d0 0x7f36d4105650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009900 tx=0x7f36bc02bc50 comp rx=0 tx=0).stop 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 -- 172.21.15.46:0/4142482150 shutdown_connections 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 --2- 172.21.15.46:0/4142482150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4105b90 0x7f36d410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 --2- 172.21.15.46:0/4142482150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d41051d0 0x7f36d4105650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 --2- 172.21.15.46:0/4142482150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4103fe0 0x7f36d4104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 -- 172.21.15.46:0/4142482150 >> 172.21.15.46:0/4142482150 conn(0x7f36d40ff570 msgr2=0x7f36d4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 -- 172.21.15.46:0/4142482150 shutdown_connections 2024-04-17T07:55:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.314+0000 7f36d3e00640 1 -- 172.21.15.46:0/4142482150 wait complete. 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.315+0000 7f36d3e00640 1 Processor -- start 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.315+0000 7f36d3e00640 1 -- start start 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4103fe0 0x7f36d419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d41051d0 0x7f36d419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4105b90 0x7f36d41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36d410fcc0 con 0x7f36d4105b90 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36d410fb40 con 0x7f36d41051d0 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36d410fe40 con 0x7f36d4103fe0 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4105b90 0x7f36d41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4105b90 0x7f36d41a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51932/0 (socket says 172.21.15.46:51932) 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 -- 172.21.15.46:0/2799983954 learned_addr learned my addr 172.21.15.46:0/2799983954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36cbe00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d41051d0 0x7f36d419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d2a00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4103fe0 0x7f36d419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 -- 172.21.15.46:0/2799983954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4103fe0 msgr2=0x7f36d419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4103fe0 0x7f36d419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 -- 172.21.15.46:0/2799983954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d41051d0 msgr2=0x7f36d419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d41051d0 0x7f36d419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.316+0000 7f36d3400640 1 -- 172.21.15.46:0/2799983954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d41a1f30 con 0x7f36d4105b90 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.317+0000 7f36d3400640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4105b90 0x7f36d41a18c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00eb10 tx=0x7f36cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.317+0000 7f36cb400640 1 -- 172.21.15.46:0/2799983954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004b20 con 0x7f36d4105b90 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.317+0000 7f36cb400640 1 -- 172.21.15.46:0/2799983954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc004cc0 con 0x7f36d4105b90 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.317+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d41a2220 con 0x7f36d4105b90 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.317+0000 7f36cb400640 1 -- 172.21.15.46:0/2799983954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc010760 con 0x7f36d4105b90 2024-04-17T07:55:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.317+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a2740 con 0x7f36d4105b90 2024-04-17T07:55:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.320+0000 7f36dc1f1740 1 Processor -- start 2024-04-17T07:55:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.320+0000 7f36cb400640 1 -- 172.21.15.46:0/2799983954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36cc005020 con 0x7f36d4105b90 2024-04-17T07:55:34.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.320+0000 7f36dc1f1740 1 -- start start 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d578ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d5793fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d5794650 0x5601d5796ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5601d56f5980 con 0x5601d578eaf0 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601d569c220 con 0x5601d5794650 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5601d56f1940 con 0x5601d578f4e0 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d578ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d578ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51934/0 (socket says 172.21.15.46:51934) 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36cbe00640 1 -- 172.21.15.46:0/1556976779 learned_addr learned my addr 172.21.15.46:0/1556976779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36d2a00640 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d5793fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.321+0000 7f36cbe00640 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d5794650 0x5601d5796ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cbe00640 1 -- 172.21.15.46:0/1556976779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 msgr2=0x5601d5793fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cbe00640 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d5793fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cbe00640 1 -- 172.21.15.46:0/1556976779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d5794650 msgr2=0x5601d5796ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cbe00640 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d5794650 0x5601d5796ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cbe00640 1 -- 172.21.15.46:0/1556976779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601d54082d0 con 0x5601d578eaf0 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36d2a00640 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d5793fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cb400640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36b80a3c30 0x7f36b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cbe00640 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d578ef10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36bc02c900 tx=0x7f36bc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0112f023030a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36cb400640 1 -- 172.21.15.46:0/2799983954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f36cc0d0b30 con 0x7f36d4105b90 2024-04-17T07:55:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36d2a00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36b80a3c30 0x7f36b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36c3400640 1 -- 172.21.15.46:0/1556976779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc049070 con 0x5601d578eaf0 2024-04-17T07:55:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36c3400640 1 -- 172.21.15.46:0/1556976779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36bc0046c0 con 0x5601d578eaf0 2024-04-17T07:55:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.322+0000 7f36c3400640 1 -- 172.21.15.46:0/1556976779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc005110 con 0x5601d578eaf0 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 -- 172.21.15.46:0/1556976779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 msgr2=0x5601d578ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36d2a00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36b80a3c30 0x7f36b80a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5601d578f8e0 tx=0x7f36c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d578ef10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36bc02c900 tx=0x7f36bc005b50 comp rx=0 tx=0).stop 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 -- 172.21.15.46:0/1556976779 shutdown_connections 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d5794650 0x5601d5796ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d5793fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/1556976779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d578ef10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 -- 172.21.15.46:0/1556976779 >> 172.21.15.46:0/1556976779 conn(0x5601d578a060 msgr2=0x5601d578a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.323+0000 7f36dc1f1740 1 -- 172.21.15.46:0/1556976779 shutdown_connections 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.324+0000 7f36dc1f1740 1 -- 172.21.15.46:0/1556976779 wait complete. 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.324+0000 7f36dc1f1740 1 -- start start 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.324+0000 7f36dc1f1740 1 Processor -- start 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.324+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578eaf0 0x5601d5799060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d57995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d57a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5601d57975a0 con 0x5601d5794650 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601d56af590 con 0x5601d578f4e0 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5601d5797720 con 0x5601d578eaf0 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36dc1f1740 10 client.0 _ll_register_callbacks cb 0x5601d5407ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d57a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d57a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51948/0 (socket says 172.21.15.46:51948) 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 -- 172.21.15.46:0/2508272793 learned_addr learned my addr 172.21.15.46:0/2508272793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d2a00640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578eaf0 0x5601d5799060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36cbe00640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d57995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 -- 172.21.15.46:0/2508272793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578eaf0 msgr2=0x5601d5799060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578eaf0 0x5601d5799060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 -- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 msgr2=0x5601d57995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d57995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 -- 172.21.15.46:0/2508272793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601d54d26b0 con 0x5601d5794650 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36cbe00640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d57995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d2a00640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578eaf0 0x5601d5799060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36d3400640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d57a05e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0995a0 tx=0x7f36cc09c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c23f15326f4625 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.325+0000 7f36b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36b7e00640 1 -- 172.21.15.46:0/2508272793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0a0f80 con 0x5601d5794650 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36b7e00640 1 -- 172.21.15.46:0/2508272793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f36cc0a53b0 con 0x5601d5794650 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36bc0046c0 con 0x5601d5794650 2024-04-17T07:55:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36b7e00640 1 -- 172.21.15.46:0/2508272793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f36cc0a5840 con 0x5601d5794650 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36dc1f1740 10 client.14574 fetch_fsmap learned FSMap version 17 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5601d57a3a20 con 0x5601d5794650 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36b7e00640 1 -- 172.21.15.46:0/2508272793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36bc0046c0 con 0x5601d5794650 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.326+0000 7f36b7e00640 1 -- 172.21.15.46:0/2508272793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) ==== 8343+0+0 (secure 0 0 0) 0x7f36cc09c020 con 0x5601d5794650 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.327+0000 7f36dc1f1740 10 client.14574 fetch_fsmap finished waiting for FSMap version 17 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.327+0000 7f36dc1f1740 10 client.14574 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.327+0000 7f36dc1f1740 20 client.14574 populate_metadata read hostname 'smithi046' 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.327+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d57a6070 0x5601d57a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.327+0000 7f36dc1f1740 4 client.14574 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.327+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601d57033a0 con 0x5601d57a6070 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.327+0000 7f36d2a00640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d57a6070 0x5601d57a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.328+0000 7f36d2a00640 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d57a6070 0x5601d57a74e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1975611c6ccda8f server_cookie=f39977daf2b58250 in_seq=0 out_seq=0 2024-04-17T07:55:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.328+0000 7f36b7e00640 10 client.14574 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:55:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36b7e00640 1 -- 172.21.15.46:0/2508272793 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601d57033a0 con 0x5601d57a6070 2024-04-17T07:55:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36b7e00640 10 client.14574 handle_command_reply: tid=0 2024-04-17T07:55:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 client.14574 shutdown 2024-04-17T07:55:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 msgr2=0x5601d57a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d57a05e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0995a0 tx=0x7f36cc09c5f0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 shutdown_connections 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d57a6070 0x5601d57a74e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d57a05e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d57995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/2508272793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578eaf0 0x5601d5799060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 >> 172.21.15.46:0/2508272793 conn(0x5601d578a060 msgr2=0x5601d57893a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36b7e00640 0 client.14574 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 shutdown_connections 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 1 -- 172.21.15.46:0/2508272793 wait complete. 2024-04-17T07:55:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.329+0000 7f36dc1f1740 20 client.14574 trim_cache size 0 max 16384 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 Processor -- start 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 -- start start 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578eaf0 0x5601d5797ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d57a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d5798000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5601d57a5820 con 0x5601d5794650 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601d57a5ef0 con 0x5601d578eaf0 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5601d57a65c0 con 0x5601d578f4e0 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d5798000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36d2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578eaf0 0x5601d5797ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d57a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36d2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578eaf0 0x5601d5797ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51522/0 (socket says 172.21.15.46:51522) 2024-04-17T07:55:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.350+0000 7f36d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d5798000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51964/0 (socket says 172.21.15.46:51964) 2024-04-17T07:55:34.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d2a00640 1 -- 172.21.15.46:0/3427689584 learned_addr learned my addr 172.21.15.46:0/3427689584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:34.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d3400640 1 -- 172.21.15.46:0/3427689584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 msgr2=0x5601d57a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d3400640 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d57a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d3400640 1 -- 172.21.15.46:0/3427689584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578eaf0 msgr2=0x5601d5797ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d3400640 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578eaf0 0x5601d5797ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d3400640 1 -- 172.21.15.46:0/3427689584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601d57858a0 con 0x5601d5794650 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d2a00640 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578eaf0 0x5601d5797ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36cbe00640 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d57a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.351+0000 7f36d3400640 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d5798000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f36cc004af0 tx=0x7f36cc09b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea2f0f83c6a3901 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36c2a00640 1 -- 172.21.15.46:0/3427689584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc07f070 con 0x5601d5794650 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36c2a00640 1 -- 172.21.15.46:0/3427689584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc0a0f80 con 0x5601d5794650 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36c2a00640 1 -- 172.21.15.46:0/3427689584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc09c020 con 0x5601d5794650 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3427689584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 msgr2=0x5601d5798000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d5798000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f36cc004af0 tx=0x7f36cc09b0e0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3427689584 shutdown_connections 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d5794650 0x5601d5798000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d578f4e0 0x5601d57a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3427689584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578eaf0 0x5601d5797ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.352+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3427689584 >> 172.21.15.46:0/3427689584 conn(0x5601d578a060 msgr2=0x5601d578a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.353+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3427689584 shutdown_connections 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.353+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3427689584 wait complete. 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.353+0000 7f36dc1f1740 1 -- start start 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.353+0000 7f36dc1f1740 1 Processor -- start 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d579c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d579cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36dc1f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d5794650 0x5601d579f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5601d57a7360 con 0x5601d578eaf0 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601d57a6c90 con 0x5601d578f4e0 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36dc1f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5601d57a7a30 con 0x5601d5794650 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36dc1f1740 10 client.0 _ll_register_callbacks cb 0x7f36cc004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d579c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d579cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d5794650 0x5601d579f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d579c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51972/0 (socket says 172.21.15.46:51972) 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d2a00640 1 -- 172.21.15.46:0/3082960913 learned_addr learned my addr 172.21.15.46:0/3082960913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d579cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51528/0 (socket says 172.21.15.46:51528) 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d3400640 1 -- 172.21.15.46:0/3082960913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 msgr2=0x5601d579cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d3400640 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d579cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d3400640 1 -- 172.21.15.46:0/3082960913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 msgr2=0x5601d579c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d3400640 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d579c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d3400640 1 -- 172.21.15.46:0/3082960913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601d579fce0 con 0x5601d5794650 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.354+0000 7f36d2a00640 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d579c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36d3400640 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d5794650 0x5601d579f6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36cc09c8d0 tx=0x7f36cc0056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91194acae297e00f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36b7e00640 1 -- 172.21.15.46:0/3082960913 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc07f030 con 0x5601d5794650 2024-04-17T07:55:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36cc0a0f80 con 0x5601d5794650 2024-04-17T07:55:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36b7e00640 1 -- 172.21.15.46:0/3082960913 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f36cc0a55b0 con 0x5601d5794650 2024-04-17T07:55:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36b7e00640 1 -- 172.21.15.46:0/3082960913 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f36cc0aa050 con 0x5601d5794650 2024-04-17T07:55:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36dc1f1740 10 client.24694 fetch_fsmap learned FSMap version 17 2024-04-17T07:55:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5601d580c500 con 0x5601d5794650 2024-04-17T07:55:34.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.355+0000 7f36b7e00640 1 -- 172.21.15.46:0/3082960913 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc0a0f80 con 0x5601d5794650 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36b7e00640 1 -- 172.21.15.46:0/3082960913 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) ==== 8343+0+0 (secure 0 0 0) 0x7f36cc0a2360 con 0x5601d5794650 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36dc1f1740 10 client.24694 fetch_fsmap finished waiting for FSMap version 17 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36dc1f1740 10 client.24694 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36dc1f1740 20 client.24694 populate_metadata read hostname 'smithi046' 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d5810c30 0x5601d580f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36dc1f1740 4 client.24694 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "status"}) -- 0x5601d56fd210 con 0x5601d5810c30 2024-04-17T07:55:34.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.356+0000 7f36d2a00640 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d5810c30 0x5601d580f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36d2a00640 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d5810c30 0x5601d580f940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96eea674c02c632f server_cookie=4fd3f1e0080ae33b in_seq=0 out_seq=0 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36b7e00640 10 client.24694 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36b7e00640 1 -- 172.21.15.46:0/3082960913 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5601d56fd210 con 0x5601d5810c30 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36b7e00640 10 client.24694 handle_command_reply: tid=0 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 client.24694 shutdown 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d5794650 msgr2=0x5601d579f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d5794650 0x5601d579f6a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36cc09c8d0 tx=0x7f36cc0056b0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 shutdown_connections 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5601d5810c30 0x5601d580f940 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5601d5794650 0x5601d579f6a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601d578f4e0 0x5601d579cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 --2- 172.21.15.46:0/3082960913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5601d578eaf0 0x5601d579c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 >> 172.21.15.46:0/3082960913 conn(0x5601d578a060 msgr2=0x5601d5797ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:34.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.357+0000 7f36b7e00640 0 client.24694 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.358+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 shutdown_connections 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.358+0000 7f36dc1f1740 1 -- 172.21.15.46:0/3082960913 wait complete. 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.358+0000 7f36dc1f1740 20 client.24694 trim_cache size 0 max 16384 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24596, 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-17T07:55:34.362 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-17T07:55:34.363 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 7.8151740099999998, 2024-04-17T07:55:34.363 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 17, 2024-04-17T07:55:34.363 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 89, 2024-04-17T07:55:34.363 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:55:34.363 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 14.319075661999999 2024-04-17T07:55:34.363 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.358+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36b80a3c30 msgr2=0x7f36b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.358+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36b80a3c30 0x7f36b80a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5601d578f8e0 tx=0x7f36c4005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.358+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4105b90 msgr2=0x7f36d41a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4105b90 0x7f36d41a18c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00eb10 tx=0x7f36cc00c2d0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 shutdown_connections 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36b80a3c30 0x7f36b80a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4105b90 0x7f36d41a18c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d41051d0 0x7f36d419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 --2- 172.21.15.46:0/2799983954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4103fe0 0x7f36d419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 >> 172.21.15.46:0/2799983954 conn(0x7f36d40ff570 msgr2=0x7f36d41084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 shutdown_connections 2024-04-17T07:55:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:34.359+0000 7f36d3e00640 1 -- 172.21.15.46:0/2799983954 wait complete. 2024-04-17T07:55:34.381 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T07:55:34.381 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:34.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph fs ls 2024-04-17T07:55:34.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:34 smithi046 ceph-mon[32038]: pgmap v325: 129 pgs: 1 creating+activating, 4 creating+peering, 123 active+clean, 1 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 828 B/s wr, 7 op/s 2024-04-17T07:55:34.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:34 smithi046 ceph-mon[32038]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-17T07:55:34.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:34 smithi046 ceph-mon[32038]: Cluster is now healthy 2024-04-17T07:55:34.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:34 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2573634173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:34.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:34 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2153486396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:34 smithi053 ceph-mon[36437]: pgmap v325: 129 pgs: 1 creating+activating, 4 creating+peering, 123 active+clean, 1 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 828 B/s wr, 7 op/s 2024-04-17T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:34 smithi053 ceph-mon[36437]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-17T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:34 smithi053 ceph-mon[36437]: Cluster is now healthy 2024-04-17T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:34 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2573634173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:34.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:34 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2153486396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:35.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:34 smithi062 ceph-mon[36507]: pgmap v325: 129 pgs: 1 creating+activating, 4 creating+peering, 123 active+clean, 1 unknown; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 828 B/s wr, 7 op/s 2024-04-17T07:55:35.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:34 smithi062 ceph-mon[36507]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-17T07:55:35.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:34 smithi062 ceph-mon[36507]: Cluster is now healthy 2024-04-17T07:55:35.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:34 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2573634173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:35.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:34 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2153486396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:36.871 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:36 smithi046 ceph-mon[32038]: pgmap v326: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-04-17T07:55:36.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:36 smithi062 ceph-mon[36507]: pgmap v326: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-04-17T07:55:36.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:36 smithi053 ceph-mon[36437]: pgmap v326: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-04-17T07:55:37.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 -- 172.21.15.46:0/2904064392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120102b00 msgr2=0x7f3120102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:37.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 --2- 172.21.15.46:0/2904064392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120102b00 0x7f3120102f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31080098f0 tx=0x7f3108009c00 comp rx=0 tx=0).stop 2024-04-17T07:55:37.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 -- 172.21.15.46:0/2904064392 shutdown_connections 2024-04-17T07:55:37.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 --2- 172.21.15.46:0/2904064392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31201034c0 0x7f312010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 --2- 172.21.15.46:0/2904064392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3120107950 0x7f3120107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 --2- 172.21.15.46:0/2904064392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3120102b00 0x7f3120102f80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 -- 172.21.15.46:0/2904064392 >> 172.21.15.46:0/2904064392 conn(0x7f3120079bb0 msgr2=0x7f3120079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:37.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 -- 172.21.15.46:0/2904064392 shutdown_connections 2024-04-17T07:55:37.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.161+0000 7f3125000700 1 -- 172.21.15.46:0/2904064392 wait complete. 2024-04-17T07:55:37.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 Processor -- start 2024-04-17T07:55:37.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 -- start start 2024-04-17T07:55:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3120102b00 0x7f312019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31201034c0 0x7f312019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3120107950 0x7f312019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3120112e10 con 0x7f3120107950 2024-04-17T07:55:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3120112c90 con 0x7f3120102b00 2024-04-17T07:55:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3125000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3120112f90 con 0x7f31201034c0 2024-04-17T07:55:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.163+0000 7f3115400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31201034c0 0x7f312019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31201034c0 0x7f312019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:39522/0 (socket says 172.21.15.46:39522) 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 -- 172.21.15.46:0/29654095 learned_addr learned my addr 172.21.15.46:0/29654095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f311e800700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3120107950 0x7f312019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f311de00700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3120102b00 0x7f312019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 -- 172.21.15.46:0/29654095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3120102b00 msgr2=0x7f312019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3120102b00 0x7f312019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 -- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3120107950 msgr2=0x7f312019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3120107950 0x7f312019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 -- 172.21.15.46:0/29654095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31201a0450 con 0x7f31201034c0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f311e800700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3120107950 0x7f312019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3115400700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31201034c0 0x7f312019b980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3108009fd0 tx=0x7f31080050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.164+0000 7f3117400700 1 -- 172.21.15.46:0/29654095 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108025070 con 0x7f31201034c0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.165+0000 7f3117400700 1 -- 172.21.15.46:0/29654095 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31080055a0 con 0x7f31201034c0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.165+0000 7f3117400700 1 -- 172.21.15.46:0/29654095 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108004e20 con 0x7f31201034c0 2024-04-17T07:55:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.165+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31201a06e0 con 0x7f31201034c0 2024-04-17T07:55:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.165+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31201a7fc0 con 0x7f31201034c0 2024-04-17T07:55:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.166+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31201041c0 con 0x7f31201034c0 2024-04-17T07:55:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.168+0000 7f3117400700 1 -- 172.21.15.46:0/29654095 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3108028070 con 0x7f31201034c0 2024-04-17T07:55:37.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.169+0000 7f3117400700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31040a3c40 0x7f31040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.169+0000 7f3117400700 1 -- 172.21.15.46:0/29654095 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f31080d0fa0 con 0x7f31201034c0 2024-04-17T07:55:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.169+0000 7f311de00700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31040a3c40 0x7f31040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.170+0000 7f311de00700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31040a3c40 0x7f31040a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f311000bbb0 tx=0x7f311000b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.173+0000 7f3117400700 1 -- 172.21.15.46:0/29654095 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f310805cff0 con 0x7f31201034c0 2024-04-17T07:55:37.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.456+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f31201a8190 con 0x7f31201034c0 2024-04-17T07:55:37.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.457+0000 7f3117400700 1 -- 172.21.15.46:0/29654095 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f3108013030 con 0x7f31201034c0 2024-04-17T07:55:37.457 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T07:55:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.460+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31040a3c40 msgr2=0x7f31040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.460+0000 7f3125000700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31040a3c40 0x7f31040a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f311000bbb0 tx=0x7f311000b590 comp rx=0 tx=0).stop 2024-04-17T07:55:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.460+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31201034c0 msgr2=0x7f312019b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.460+0000 7f3125000700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31201034c0 0x7f312019b980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3108009fd0 tx=0x7f31080050f0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 shutdown_connections 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f31040a3c40 0x7f31040a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3120107950 0x7f312019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3120102b00 0x7f312019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 --2- 172.21.15.46:0/29654095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31201034c0 0x7f312019b980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 >> 172.21.15.46:0/29654095 conn(0x7f3120079bb0 msgr2=0x7f3120100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 shutdown_connections 2024-04-17T07:55:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:37.461+0000 7f3125000700 1 -- 172.21.15.46:0/29654095 wait complete. 2024-04-17T07:55:37.842 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:37 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/29654095' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:55:37.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:37 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/29654095' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:55:38.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:37 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/29654095' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T07:55:38.208 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-17T07:55:38.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:38.209 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-17T07:55:38.268 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T07:55:38.268 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T07:55:38.268 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T07:55:38.268 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:55:38.268 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-17T07:55:38.268 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:55:38.268 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41984sec preferred_lft 41984sec 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-17T07:55:38.269 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:55:38.270 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:55:38.270 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:55:38.270 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-17T07:55:38.271 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T07:55:38.271 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:55:38.271 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:55:38.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-04-17T07:55:38.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-04-17T07:55:38.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-04-17T07:55:38.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T07:55:38.272 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:55:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:55:38.678 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:38 smithi046 ceph-mon[32038]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 386 B/s wr, 2 op/s 2024-04-17T07:55:38.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:55:38.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:38.834 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T07:55:38.920 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-17T07:55:38.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:38.924 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-17T07:55:38.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:38 smithi053 ceph-mon[36437]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 386 B/s wr, 2 op/s 2024-04-17T07:55:38.990 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-17T07:55:38.991 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T07:55:38.991 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-17T07:55:38.991 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-17T07:55:38.991 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T07:55:38.992 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:55:38.992 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:55:38.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T07:55:38.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T07:55:38.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T07:55:38.992 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:55:39.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:38 smithi062 ceph-mon[36507]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 386 B/s wr, 2 op/s 2024-04-17T07:55:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:55:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:55:39.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:39.849 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-17T07:55:39.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:39.907 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-17T07:55:39.963 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:55:39.963 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:55:39.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:55:39.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-17T07:55:39.963 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:55:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:55:40.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:55:40.189 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-17T07:55:40.189 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:55:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:55:40.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:55:40.556 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:55:40.556 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:55:40.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-17T07:55:40.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T07:55:40.556 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:55:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:55:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:55:40.796 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-17T07:55:40.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:55:40.797 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:40.842 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-17T07:55:40.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:55:40.842 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:40.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-17T07:55:40.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:40 smithi053 ceph-mon[36437]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 229 B/s wr, 1 op/s 2024-04-17T07:55:41.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:40 smithi046 ceph-mon[32038]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 229 B/s wr, 1 op/s 2024-04-17T07:55:41.034 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:55:41.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:40 smithi062 ceph-mon[36507]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 229 B/s wr, 1 op/s 2024-04-17T07:55:41.091 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-17T07:55:41.091 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-17T07:55:41.091 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-17T07:55:41.091 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-17T07:55:41.091 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-17T07:55:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/549e5c6b6d0c8ba4eefa8ba569122ceac901a22b3aa261ef34b9164f431574b4/merged 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4a51a4c42a59067ee6a421f3a2b932c99c2324e4d7a2c4e62026171acefea210/merged 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/57a9616f514eae9b3b407c4f29a938a7f5f87a1a9038a8309a1222a69ec800af/merged 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/63401cd2c4e38c977f344cb8ca62718f15bf03ddb239d181fe22b7000e69dd76/merged 2024-04-17T07:55:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/1bb2aea8ef6abe4e029133a06ab3e883a33c77fcba0dc4a2c81145075d7744ce/merged 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/172bf9971754b0a399b19a9dfb16031d83211711872857f842e5c06ce6aba6e1/merged 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6f19c3d88d89b89ac0236ec66042944f546af9db10667af442fc8acb3f893356/merged 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a62450a248badf532ea66db886a8ced577be4b4fc8097626f15940351a21eb0e/merged 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/857ce3a4cda9aacedc37e1aa5acda42ae811ff2944053cb34f0765c99aad607a/merged 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/5f0596dbcd708bb7ea400de1bcc80f2394c352278ba852be042ec3c960e42a84/merged 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:55:41.094 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:55:41.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:41.095 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-17T07:55:41.142 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T07:55:41.142 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-17T07:55:41.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-17T07:55:41.275 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:55:41.324 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-17T07:55:41.324 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-17T07:55:41.324 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-17T07:55:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/549e5c6b6d0c8ba4eefa8ba569122ceac901a22b3aa261ef34b9164f431574b4/merged 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4a51a4c42a59067ee6a421f3a2b932c99c2324e4d7a2c4e62026171acefea210/merged 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:55:41.326 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/57a9616f514eae9b3b407c4f29a938a7f5f87a1a9038a8309a1222a69ec800af/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/63401cd2c4e38c977f344cb8ca62718f15bf03ddb239d181fe22b7000e69dd76/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/1bb2aea8ef6abe4e029133a06ab3e883a33c77fcba0dc4a2c81145075d7744ce/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/172bf9971754b0a399b19a9dfb16031d83211711872857f842e5c06ce6aba6e1/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6f19c3d88d89b89ac0236ec66042944f546af9db10667af442fc8acb3f893356/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a62450a248badf532ea66db886a8ced577be4b4fc8097626f15940351a21eb0e/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/857ce3a4cda9aacedc37e1aa5acda42ae811ff2944053cb34f0765c99aad607a/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/5f0596dbcd708bb7ea400de1bcc80f2394c352278ba852be042ec3c960e42a84/merged 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:55:41.327 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:55:41.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:41.328 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-17T07:55:41.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:2024-04-17T07:55:41.412+0000 7fbef2838480 -1 init, newargv = 0x5581e1d381a0 newargc=15 2024-04-17T07:55:41.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[76003]: starting ceph client 2024-04-17T07:55:41.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[76003]: starting fuse 2024-04-17T07:55:42.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-17T07:55:42.564 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:55:42.617 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-04-17T07:55:42.618 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:55:42.619 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/549e5c6b6d0c8ba4eefa8ba569122ceac901a22b3aa261ef34b9164f431574b4/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4a51a4c42a59067ee6a421f3a2b932c99c2324e4d7a2c4e62026171acefea210/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/57a9616f514eae9b3b407c4f29a938a7f5f87a1a9038a8309a1222a69ec800af/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/63401cd2c4e38c977f344cb8ca62718f15bf03ddb239d181fe22b7000e69dd76/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/1bb2aea8ef6abe4e029133a06ab3e883a33c77fcba0dc4a2c81145075d7744ce/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/172bf9971754b0a399b19a9dfb16031d83211711872857f842e5c06ce6aba6e1/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/6f19c3d88d89b89ac0236ec66042944f546af9db10667af442fc8acb3f893356/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/a62450a248badf532ea66db886a8ced577be4b4fc8097626f15940351a21eb0e/merged 2024-04-17T07:55:42.620 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/857ce3a4cda9aacedc37e1aa5acda42ae811ff2944053cb34f0765c99aad607a/merged 2024-04-17T07:55:42.621 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/5f0596dbcd708bb7ea400de1bcc80f2394c352278ba852be042ec3c960e42a84/merged 2024-04-17T07:55:42.621 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-17T07:55:42.621 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:55:42.621 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:55:42.621 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:42.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:42.622 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-17T07:55:42.673 INFO:teuthology.orchestra.run.smithi046.stdout:119 2024-04-17T07:55:42.674 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-17T07:55:42.674 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-04-17T07:55:42.675 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-04-17T07:55:42.676 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T07:55:42.676 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-17T07:55:42.676 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.admin")) 2024-04-17T07:55:42.676 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-17T07:55:42.834 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.admin.76003.asok 2024-04-17T07:55:42.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:55:42.848 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:42 smithi046 ceph-mon[32038]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:42.852 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76003.asok 2024-04-17T07:55:42.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:42.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76003.asok status 2024-04-17T07:55:42.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:42 smithi053 ceph-mon[36437]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:43.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:42 smithi062 ceph-mon[36507]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:55:43.098 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:55:43.098 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "admin", 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "76003", 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 14592, 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-04-17T07:55:43.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 14592 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3604308384 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3604308384 2024-04-17T07:55:43.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.14592 172.21.15.46:0/3604308384", 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "172.21.15.46:0/3604308384", 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 17, 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 89, 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-04-17T07:55:43.101 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:55:43.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:43.114 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-04-17T07:55:43.114 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:43.173 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-04-17T07:55:43.174 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:43.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:43.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:43.279 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-17T07:55:43.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T07:55:43.433 INFO:teuthology.orchestra.run.smithi046.stdout:# file: . 2024-04-17T07:55:43.433 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-17T07:55:43.433 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:55:43.444 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:55:43.444 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:55:43.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.709+0000 7f9a9b200640 1 -- 172.21.15.46:0/1032486105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a94075fa0 msgr2=0x7f9a940743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.709+0000 7f9a9b200640 1 --2- 172.21.15.46:0/1032486105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a94075fa0 0x7f9a940743f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009900 tx=0x7f9a8402bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.710+0000 7f9a9b200640 1 -- 172.21.15.46:0/1032486105 shutdown_connections 2024-04-17T07:55:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.710+0000 7f9a9b200640 1 --2- 172.21.15.46:0/1032486105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a94109ee0 0x7f9a9410c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.710+0000 7f9a9b200640 1 --2- 172.21.15.46:0/1032486105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a940749c0 0x7f9a94074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.710+0000 7f9a9b200640 1 --2- 172.21.15.46:0/1032486105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a94075fa0 0x7f9a940743f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.710+0000 7f9a9b200640 1 -- 172.21.15.46:0/1032486105 >> 172.21.15.46:0/1032486105 conn(0x7f9a940fd590 msgr2=0x7f9a940ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:43.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.710+0000 7f9a9b200640 1 -- 172.21.15.46:0/1032486105 shutdown_connections 2024-04-17T07:55:43.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.710+0000 7f9a9b200640 1 -- 172.21.15.46:0/1032486105 wait complete. 2024-04-17T07:55:43.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.711+0000 7f9a9b200640 1 Processor -- start 2024-04-17T07:55:43.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.711+0000 7f9a9b200640 1 -- start start 2024-04-17T07:55:43.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a9b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a940749c0 0x7f9a9419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:43.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a9b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a94075fa0 0x7f9a9419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:43.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a9b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a94109ee0 0x7f9a941a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:43.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a9b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9410fe00 con 0x7f9a940749c0 2024-04-17T07:55:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a9b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9410fc80 con 0x7f9a94109ee0 2024-04-17T07:55:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a9b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9410ff80 con 0x7f9a94075fa0 2024-04-17T07:55:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a940749c0 0x7f9a9419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a940749c0 0x7f9a9419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42588/0 (socket says 172.21.15.46:42588) 2024-04-17T07:55:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a93e00640 1 -- 172.21.15.46:0/791475622 learned_addr learned my addr 172.21.15.46:0/791475622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a99200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a94109ee0 0x7f9a941a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.712+0000 7f9a93400640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a94075fa0 0x7f9a9419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a99200640 1 -- 172.21.15.46:0/791475622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a94075fa0 msgr2=0x7f9a9419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a99200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a94075fa0 0x7f9a9419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a99200640 1 -- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a940749c0 msgr2=0x7f9a9419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a99200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a940749c0 0x7f9a9419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a99200640 1 -- 172.21.15.46:0/791475622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a941a20c0 con 0x7f9a94109ee0 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a99200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a94109ee0 0x7f9a941a1a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009fd0 tx=0x7f9a8c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a90c00640 1 -- 172.21.15.46:0/791475622 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00ce20 con 0x7f9a94109ee0 2024-04-17T07:55:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a941a23b0 con 0x7f9a94109ee0 2024-04-17T07:55:43.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a90c00640 1 -- 172.21.15.46:0/791475622 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c005020 con 0x7f9a94109ee0 2024-04-17T07:55:43.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a90c00640 1 -- 172.21.15.46:0/791475622 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c010720 con 0x7f9a94109ee0 2024-04-17T07:55:43.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.713+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a941a28b0 con 0x7f9a94109ee0 2024-04-17T07:55:43.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.715+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a94108c10 con 0x7f9a94109ee0 2024-04-17T07:55:43.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.716+0000 7f9a90c00640 1 -- 172.21.15.46:0/791475622 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9a8c005540 con 0x7f9a94109ee0 2024-04-17T07:55:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.718+0000 7f9a90c00640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a780a3bb0 0x7f9a780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.718+0000 7f9a90c00640 1 -- 172.21.15.46:0/791475622 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9a8c0d0d50 con 0x7f9a94109ee0 2024-04-17T07:55:43.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.718+0000 7f9a93e00640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a780a3bb0 0x7f9a780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:43.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.719+0000 7f9a93e00640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a780a3bb0 0x7f9a780a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a84000c00 tx=0x7f9a8402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:43.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.723+0000 7f9a90c00640 1 -- 172.21.15.46:0/791475622 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9a8c09d490 con 0x7f9a94109ee0 2024-04-17T07:55:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.933+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a94062b60 con 0x7f9a94109ee0 2024-04-17T07:55:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.936+0000 7f9a90c00640 1 -- 172.21.15.46:0/791475622 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f9a8c09cd90 con 0x7f9a94109ee0 2024-04-17T07:55:43.935 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a780a3bb0 msgr2=0x7f9a780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a780a3bb0 0x7f9a780a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a84000c00 tx=0x7f9a8402c880 comp rx=0 tx=0).stop 2024-04-17T07:55:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a94109ee0 msgr2=0x7f9a941a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a94109ee0 0x7f9a941a1a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009fd0 tx=0x7f9a8c00ef10 comp rx=0 tx=0).stop 2024-04-17T07:55:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 shutdown_connections 2024-04-17T07:55:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a780a3bb0 0x7f9a780a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a94109ee0 0x7f9a941a1a50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a94075fa0 0x7f9a9419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.938+0000 7f9a9b200640 1 --2- 172.21.15.46:0/791475622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a940749c0 0x7f9a9419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.939+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 >> 172.21.15.46:0/791475622 conn(0x7f9a940fd590 msgr2=0x7f9a94105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.939+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 shutdown_connections 2024-04-17T07:55:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:43.939+0000 7f9a9b200640 1 -- 172.21.15.46:0/791475622 wait complete. 2024-04-17T07:55:43.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:55:43.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:55:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.191+0000 7f830be00640 1 -- 172.21.15.46:0/3793875597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c1051c0 msgr2=0x7f830c105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.191+0000 7f830be00640 1 --2- 172.21.15.46:0/3793875597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c1051c0 0x7f830c105640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009860 tx=0x7f82f402bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.192+0000 7f830be00640 1 -- 172.21.15.46:0/3793875597 shutdown_connections 2024-04-17T07:55:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.192+0000 7f830be00640 1 --2- 172.21.15.46:0/3793875597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c105b80 0x7f830c10c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.192+0000 7f830be00640 1 --2- 172.21.15.46:0/3793875597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c1051c0 0x7f830c105640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.192+0000 7f830be00640 1 --2- 172.21.15.46:0/3793875597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f830c103fd0 0x7f830c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.192+0000 7f830be00640 1 -- 172.21.15.46:0/3793875597 >> 172.21.15.46:0/3793875597 conn(0x7f830c0ff540 msgr2=0x7f830c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:44.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.192+0000 7f830be00640 1 -- 172.21.15.46:0/3793875597 shutdown_connections 2024-04-17T07:55:44.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.192+0000 7f830be00640 1 -- 172.21.15.46:0/3793875597 wait complete. 2024-04-17T07:55:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.193+0000 7f830be00640 1 Processor -- start 2024-04-17T07:55:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.193+0000 7f830be00640 1 -- start start 2024-04-17T07:55:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f830c103fd0 0x7f830c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c1051c0 0x7f830c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c105b80 0x7f830c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10fc60 con 0x7f830c1051c0 2024-04-17T07:55:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10fae0 con 0x7f830c105b80 2024-04-17T07:55:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10fde0 con 0x7f830c103fd0 2024-04-17T07:55:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c105b80 0x7f830c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c1051c0 0x7f830c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f830c103fd0 0x7f830c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c105b80 0x7f830c1a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58086/0 (socket says 172.21.15.46:58086) 2024-04-17T07:55:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.194+0000 7f830b400640 1 -- 172.21.15.46:0/945736584 learned_addr learned my addr 172.21.15.46:0/945736584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830b400640 1 -- 172.21.15.46:0/945736584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f830c103fd0 msgr2=0x7f830c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830b400640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f830c103fd0 0x7f830c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830b400640 1 -- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c1051c0 msgr2=0x7f830c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830b400640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c1051c0 0x7f830c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830b400640 1 -- 172.21.15.46:0/945736584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c1a1e60 con 0x7f830c105b80 2024-04-17T07:55:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830b400640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c105b80 0x7f830c1a17f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f830400eb10 tx=0x7f830400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f8303400640 1 -- 172.21.15.46:0/945736584 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304004b20 con 0x7f830c105b80 2024-04-17T07:55:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f8303400640 1 -- 172.21.15.46:0/945736584 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8304004cc0 con 0x7f830c105b80 2024-04-17T07:55:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c1a2150 con 0x7f830c105b80 2024-04-17T07:55:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f8303400640 1 -- 172.21.15.46:0/945736584 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304010760 con 0x7f830c105b80 2024-04-17T07:55:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.195+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a2620 con 0x7f830c105b80 2024-04-17T07:55:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.197+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c10b290 con 0x7f830c105b80 2024-04-17T07:55:44.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.198+0000 7f8303400640 1 -- 172.21.15.46:0/945736584 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8304005020 con 0x7f830c105b80 2024-04-17T07:55:44.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.200+0000 7f8303400640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82f00a3ba0 0x7f82f00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.200+0000 7f8303400640 1 -- 172.21.15.46:0/945736584 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f83040d0c10 con 0x7f830c105b80 2024-04-17T07:55:44.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.200+0000 7f830aa00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82f00a3ba0 0x7f82f00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.201+0000 7f830aa00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82f00a3ba0 0x7f82f00a6060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009530 tx=0x7f82fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:44.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.204+0000 7f8303400640 1 -- 172.21.15.46:0/945736584 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f830409d350 con 0x7f830c105b80 2024-04-17T07:55:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.411+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f830c062b60 con 0x7f830c105b80 2024-04-17T07:55:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.411+0000 7f8303400640 1 -- 172.21.15.46:0/945736584 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f830409cc50 con 0x7f830c105b80 2024-04-17T07:55:44.411 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82f00a3ba0 msgr2=0x7f82f00a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82f00a3ba0 0x7f82f00a6060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009530 tx=0x7f82fc009260 comp rx=0 tx=0).stop 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c105b80 msgr2=0x7f830c1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c105b80 0x7f830c1a17f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f830400eb10 tx=0x7f830400c2d0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 shutdown_connections 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82f00a3ba0 0x7f82f00a6060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f830c105b80 0x7f830c1a17f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c1051c0 0x7f830c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 --2- 172.21.15.46:0/945736584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f830c103fd0 0x7f830c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.414+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 >> 172.21.15.46:0/945736584 conn(0x7f830c0ff540 msgr2=0x7f830c108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.415+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 shutdown_connections 2024-04-17T07:55:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.415+0000 7f830be00640 1 -- 172.21.15.46:0/945736584 wait complete. 2024-04-17T07:55:44.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 -- 172.21.15.46:0/1777075844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4075f40 msgr2=0x7efec4074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 --2- 172.21.15.46:0/1777075844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4075f40 0x7efec4074390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009ca0 tx=0x7efeb802c190 comp rx=0 tx=0).stop 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 -- 172.21.15.46:0/1777075844 shutdown_connections 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 --2- 172.21.15.46:0/1777075844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec4109e40 0x7efec410c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 --2- 172.21.15.46:0/1777075844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4074960 0x7efec4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 --2- 172.21.15.46:0/1777075844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4075f40 0x7efec4074390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 -- 172.21.15.46:0/1777075844 >> 172.21.15.46:0/1777075844 conn(0x7efec40fd520 msgr2=0x7efec40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.660+0000 7efecbe00640 1 -- 172.21.15.46:0/1777075844 shutdown_connections 2024-04-17T07:55:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.661+0000 7efecbe00640 1 -- 172.21.15.46:0/1777075844 wait complete. 2024-04-17T07:55:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.661+0000 7efecbe00640 1 Processor -- start 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efecbe00640 1 -- start start 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4074960 0x7efec419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efecbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4109e40 0x7efec419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efecbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec419fc80 0x7efec41a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efecbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efec410fe40 con 0x7efec4074960 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efecbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efec410fcc0 con 0x7efec4109e40 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efecbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efec410ffc0 con 0x7efec419fc80 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efec9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4074960 0x7efec419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efec9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4074960 0x7efec419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42622/0 (socket says 172.21.15.46:42622) 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efec9400640 1 -- 172.21.15.46:0/294202421 learned_addr learned my addr 172.21.15.46:0/294202421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.662+0000 7efec8a00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4109e40 0x7efec419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec9e00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec419fc80 0x7efec41a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec9400640 1 -- 172.21.15.46:0/294202421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec419fc80 msgr2=0x7efec41a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec9400640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec419fc80 0x7efec41a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec9400640 1 -- 172.21.15.46:0/294202421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4109e40 msgr2=0x7efec419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec9400640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4109e40 0x7efec419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec9400640 1 -- 172.21.15.46:0/294202421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41a26e0 con 0x7efec4074960 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec9400640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4074960 0x7efec419a6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009bf0 tx=0x7efeb8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec2000640 1 -- 172.21.15.46:0/294202421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8039040 con 0x7efec4074960 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec2000640 1 -- 172.21.15.46:0/294202421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb8003e50 con 0x7efec4074960 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41a2970 con 0x7efec4074960 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.663+0000 7efec2000640 1 -- 172.21.15.46:0/294202421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80042a0 con 0x7efec4074960 2024-04-17T07:55:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.664+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41a2eb0 con 0x7efec4074960 2024-04-17T07:55:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.665+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec4108c00 con 0x7efec4074960 2024-04-17T07:55:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.667+0000 7efec2000640 1 -- 172.21.15.46:0/294202421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efeb8046030 con 0x7efec4074960 2024-04-17T07:55:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.668+0000 7efec2000640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efeac0a3c30 0x7efeac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.668+0000 7efec2000640 1 -- 172.21.15.46:0/294202421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efeb80f22b0 con 0x7efec4074960 2024-04-17T07:55:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.669+0000 7efec8a00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efeac0a3c30 0x7efeac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.669+0000 7efec8a00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efeac0a3c30 0x7efeac0a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efeb00095f0 tx=0x7efeb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.671+0000 7efec2000640 1 -- 172.21.15.46:0/294202421 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efeb804d060 con 0x7efec4074960 2024-04-17T07:55:44.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:44 smithi046 ceph-mon[32038]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-04-17T07:55:44.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:44 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/791475622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:44.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:44 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/945736584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:44.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.879+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efec402d7f0 con 0x7efec4074960 2024-04-17T07:55:44.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.880+0000 7efec2000640 1 -- 172.21.15.46:0/294202421 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7efeb80beb10 con 0x7efec4074960 2024-04-17T07:55:44.880 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efeac0a3c30 msgr2=0x7efeac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efeac0a3c30 0x7efeac0a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efeb00095f0 tx=0x7efeb0005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4074960 msgr2=0x7efec419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4074960 0x7efec419a6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009bf0 tx=0x7efeb8009360 comp rx=0 tx=0).stop 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 shutdown_connections 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efeac0a3c30 0x7efeac0a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efec419fc80 0x7efec41a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4109e40 0x7efec419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 --2- 172.21.15.46:0/294202421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec4074960 0x7efec419a6e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.883+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 >> 172.21.15.46:0/294202421 conn(0x7efec40fd520 msgr2=0x7efec4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.884+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 shutdown_connections 2024-04-17T07:55:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:44.884+0000 7efecbe00640 1 -- 172.21.15.46:0/294202421 wait complete. 2024-04-17T07:55:44.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:55:44.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:44 smithi053 ceph-mon[36437]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-04-17T07:55:44.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:44 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/791475622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:44.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:44 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/945736584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:44 smithi062 ceph-mon[36507]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-04-17T07:55:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:44 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/791475622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:45.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:44 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/945736584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:45.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 -- 172.21.15.46:0/1741550332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 msgr2=0x7f1f8410a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 --2- 172.21.15.46:0/1741550332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 0x7f1f8410a1e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009ca0 tx=0x7f1f7c02c100 comp rx=0 tx=0).stop 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 -- 172.21.15.46:0/1741550332 shutdown_connections 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 --2- 172.21.15.46:0/1741550332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 0x7f1f8410a1e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 --2- 172.21.15.46:0/1741550332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f84102f90 0x7f1f84103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 --2- 172.21.15.46:0/1741550332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 0x7f1f841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 -- 172.21.15.46:0/1741550332 >> 172.21.15.46:0/1741550332 conn(0x7f1f840fd310 msgr2=0x7f1f840ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 -- 172.21.15.46:0/1741550332 shutdown_connections 2024-04-17T07:55:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.137+0000 7f1f83e00640 1 -- 172.21.15.46:0/1741550332 wait complete. 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.138+0000 7f1f83e00640 1 Processor -- start 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.138+0000 7f1f83e00640 1 -- start start 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 0x7f1f8419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f84102f90 0x7f1f8419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 0x7f1f841a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f83e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8410da00 con 0x7f1f84103950 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f83e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8410d880 con 0x7f1f84102f90 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f83e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8410db80 con 0x7f1f84101da0 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f82a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 0x7f1f8419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f83400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 0x7f1f841a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f82a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 0x7f1f8419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37688/0 (socket says 172.21.15.46:37688) 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f82a00640 1 -- 172.21.15.46:0/575336736 learned_addr learned my addr 172.21.15.46:0/575336736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f82000640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f84102f90 0x7f1f8419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f82a00640 1 -- 172.21.15.46:0/575336736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f84102f90 msgr2=0x7f1f8419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.139+0000 7f1f82a00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f84102f90 0x7f1f8419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f82a00640 1 -- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 msgr2=0x7f1f841a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f82a00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 0x7f1f841a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f82a00640 1 -- 172.21.15.46:0/575336736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f841a1ec0 con 0x7f1f84101da0 2024-04-17T07:55:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f82a00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 0x7f1f8419a2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400e9c0 tx=0x7f1f7400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f88a00640 1 -- 172.21.15.46:0/575336736 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400cd80 con 0x7f1f84101da0 2024-04-17T07:55:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f88a00640 1 -- 172.21.15.46:0/575336736 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f740035f0 con 0x7f1f84101da0 2024-04-17T07:55:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f88a00640 1 -- 172.21.15.46:0/575336736 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74010690 con 0x7f1f84101da0 2024-04-17T07:55:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f841a21b0 con 0x7f1f84101da0 2024-04-17T07:55:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.140+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f841a2750 con 0x7f1f84101da0 2024-04-17T07:55:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.141+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f841a2a10 con 0x7f1f84101da0 2024-04-17T07:55:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.143+0000 7f1f88a00640 1 -- 172.21.15.46:0/575336736 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f740037a0 con 0x7f1f84101da0 2024-04-17T07:55:45.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.145+0000 7f1f88a00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1f680a3c20 0x7f1f680a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.145+0000 7f1f88a00640 1 -- 172.21.15.46:0/575336736 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1f740d0a00 con 0x7f1f84101da0 2024-04-17T07:55:45.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.145+0000 7f1f82000640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1f680a3c20 0x7f1f680a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.146+0000 7f1f82000640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1f680a3c20 0x7f1f680a60e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f8419b7b0 tx=0x7f1f70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.150+0000 7f1f88a00640 1 -- 172.21.15.46:0/575336736 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1f7409d140 con 0x7f1f84101da0 2024-04-17T07:55:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.356+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f841a2be0 con 0x7f1f84101da0 2024-04-17T07:55:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.357+0000 7f1f88a00640 1 -- 172.21.15.46:0/575336736 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f1f7409ca40 con 0x7f1f84101da0 2024-04-17T07:55:45.356 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.359+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1f680a3c20 msgr2=0x7f1f680a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.359+0000 7f1f83e00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1f680a3c20 0x7f1f680a60e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f8419b7b0 tx=0x7f1f70005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.359+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 msgr2=0x7f1f8419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.359+0000 7f1f83e00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 0x7f1f8419a2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400e9c0 tx=0x7f1f7400ee80 comp rx=0 tx=0).stop 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.359+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 shutdown_connections 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.360+0000 7f1f83e00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1f680a3c20 0x7f1f680a60e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.360+0000 7f1f83e00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f84103950 0x7f1f841a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.360+0000 7f1f83e00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f84102f90 0x7f1f8419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.360+0000 7f1f83e00640 1 --2- 172.21.15.46:0/575336736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f84101da0 0x7f1f8419a2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.360+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 >> 172.21.15.46:0/575336736 conn(0x7f1f840fd310 msgr2=0x7f1f84074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:45.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.360+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 shutdown_connections 2024-04-17T07:55:45.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.360+0000 7f1f83e00640 1 -- 172.21.15.46:0/575336736 wait complete. 2024-04-17T07:55:45.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:55:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.611+0000 7f63a1e00640 1 -- 172.21.15.46:0/1427958906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 msgr2=0x7f639c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.611+0000 7f63a1e00640 1 --2- 172.21.15.46:0/1427958906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 0x7f639c101dc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6394009be0 tx=0x7f639402bef0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.611+0000 7f63a1e00640 1 -- 172.21.15.46:0/1427958906 shutdown_connections 2024-04-17T07:55:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.611+0000 7f63a1e00640 1 --2- 172.21.15.46:0/1427958906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 0x7f639c101dc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.611+0000 7f63a1e00640 1 --2- 172.21.15.46:0/1427958906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c100fa0 0x7f639c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.611+0000 7f63a1e00640 1 --2- 172.21.15.46:0/1427958906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 0x7f639c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.611+0000 7f63a1e00640 1 -- 172.21.15.46:0/1427958906 >> 172.21.15.46:0/1427958906 conn(0x7f639c0fc240 msgr2=0x7f639c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.612+0000 7f63a1e00640 1 -- 172.21.15.46:0/1427958906 shutdown_connections 2024-04-17T07:55:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.612+0000 7f63a1e00640 1 -- 172.21.15.46:0/1427958906 wait complete. 2024-04-17T07:55:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 Processor -- start 2024-04-17T07:55:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 -- start start 2024-04-17T07:55:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 0x7f639c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c100fa0 0x7f639c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 0x7f639c1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10fc70 con 0x7f639c100640 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10faf0 con 0x7f639c101940 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f639c10fdf0 con 0x7f639c100fa0 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.613+0000 7f63a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 0x7f639c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 0x7f639c19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42664/0 (socket says 172.21.15.46:42664) 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 -- 172.21.15.46:0/4161262014 learned_addr learned my addr 172.21.15.46:0/4161262014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f639be00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c100fa0 0x7f639c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a1400640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 0x7f639c1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 -- 172.21.15.46:0/4161262014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c100fa0 msgr2=0x7f639c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c100fa0 0x7f639c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 -- 172.21.15.46:0/4161262014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 msgr2=0x7f639c1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 0x7f639c1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 -- 172.21.15.46:0/4161262014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c1a1f00 con 0x7f639c100640 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.614+0000 7f63a0a00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 0x7f639c19a2e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9c0 tx=0x7f638c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.615+0000 7f6399600640 1 -- 172.21.15.46:0/4161262014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c00cde0 con 0x7f639c100640 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.615+0000 7f6399600640 1 -- 172.21.15.46:0/4161262014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638c0035f0 con 0x7f639c100640 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.615+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c1a21f0 con 0x7f639c100640 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.615+0000 7f6399600640 1 -- 172.21.15.46:0/4161262014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c010690 con 0x7f639c100640 2024-04-17T07:55:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.615+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a2790 con 0x7f639c100640 2024-04-17T07:55:45.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.616+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c10b390 con 0x7f639c100640 2024-04-17T07:55:45.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.618+0000 7f6399600640 1 -- 172.21.15.46:0/4161262014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f638c0037a0 con 0x7f639c100640 2024-04-17T07:55:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.620+0000 7f6399600640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63800a3c30 0x7f63800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.620+0000 7f6399600640 1 -- 172.21.15.46:0/4161262014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f638c0d08e0 con 0x7f639c100640 2024-04-17T07:55:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.620+0000 7f639be00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63800a3c30 0x7f63800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.621+0000 7f639be00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63800a3c30 0x7f63800a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f639c19b7f0 tx=0x7f6384005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.624+0000 7f6399600640 1 -- 172.21.15.46:0/4161262014 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f638c09d020 con 0x7f639c100640 2024-04-17T07:55:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.830+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f639c062b60 con 0x7f639c100640 2024-04-17T07:55:45.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.831+0000 7f6399600640 1 -- 172.21.15.46:0/4161262014 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f638c09c920 con 0x7f639c100640 2024-04-17T07:55:45.832 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:55:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63800a3c30 msgr2=0x7f63800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63800a3c30 0x7f63800a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f639c19b7f0 tx=0x7f6384005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 msgr2=0x7f639c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 0x7f639c19a2e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9c0 tx=0x7f638c00ee80 comp rx=0 tx=0).stop 2024-04-17T07:55:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 shutdown_connections 2024-04-17T07:55:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63800a3c30 0x7f63800a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f639c101940 0x7f639c1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639c100fa0 0x7f639c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 --2- 172.21.15.46:0/4161262014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f639c100640 0x7f639c19a2e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 >> 172.21.15.46:0/4161262014 conn(0x7f639c0fc240 msgr2=0x7f639c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 shutdown_connections 2024-04-17T07:55:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:45.834+0000 7f63a1e00640 1 -- 172.21.15.46:0/4161262014 wait complete. 2024-04-17T07:55:45.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T07:55:45.941 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:45 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/294202421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:55:45.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:45 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/575336736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:55:45.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:45 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/294202421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:55:45.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:45 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/575336736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:55:46.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:45 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/294202421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:55:46.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:45 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/575336736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.083+0000 7f9353c00640 1 -- 172.21.15.46:0/384959403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c1053b0 msgr2=0x7f934c105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.083+0000 7f9353c00640 1 --2- 172.21.15.46:0/384959403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c1053b0 0x7f934c105830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9334009900 tx=0x7f933402bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.083+0000 7f9353c00640 1 -- 172.21.15.46:0/384959403 shutdown_connections 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.083+0000 7f9353c00640 1 --2- 172.21.15.46:0/384959403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c105d70 0x7f934c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.083+0000 7f9353c00640 1 --2- 172.21.15.46:0/384959403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c1053b0 0x7f934c105830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.083+0000 7f9353c00640 1 --2- 172.21.15.46:0/384959403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c1041c0 0x7f934c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.083+0000 7f9353c00640 1 -- 172.21.15.46:0/384959403 >> 172.21.15.46:0/384959403 conn(0x7f934c0ff750 msgr2=0x7f934c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.084+0000 7f9353c00640 1 -- 172.21.15.46:0/384959403 shutdown_connections 2024-04-17T07:55:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.084+0000 7f9353c00640 1 -- 172.21.15.46:0/384959403 wait complete. 2024-04-17T07:55:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 Processor -- start 2024-04-17T07:55:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 -- start start 2024-04-17T07:55:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c1041c0 0x7f934c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c1053b0 0x7f934c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c105d70 0x7f934c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f934c10fe50 con 0x7f934c1053b0 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f934c10fcd0 con 0x7f934c105d70 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9353c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f934c10ffd0 con 0x7f934c1041c0 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f934be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c1053b0 0x7f934c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f934be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c1053b0 0x7f934c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42690/0 (socket says 172.21.15.46:42690) 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f934be00640 1 -- 172.21.15.46:0/2056851317 learned_addr learned my addr 172.21.15.46:0/2056851317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.085+0000 7f9351200640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c1041c0 0x7f934c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f9351c00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c105d70 0x7f934c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f934be00640 1 -- 172.21.15.46:0/2056851317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c1041c0 msgr2=0x7f934c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f934be00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c1041c0 0x7f934c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f934be00640 1 -- 172.21.15.46:0/2056851317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c105d70 msgr2=0x7f934c1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f934be00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c105d70 0x7f934c1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f934be00640 1 -- 172.21.15.46:0/2056851317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c1a20b0 con 0x7f934c1053b0 2024-04-17T07:55:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f934be00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c1053b0 0x7f934c19aa20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9334009230 tx=0x7f933402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.086+0000 7f9349600640 1 -- 172.21.15.46:0/2056851317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334041070 con 0x7f934c1053b0 2024-04-17T07:55:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.087+0000 7f9349600640 1 -- 172.21.15.46:0/2056851317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9334004450 con 0x7f934c1053b0 2024-04-17T07:55:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.087+0000 7f9349600640 1 -- 172.21.15.46:0/2056851317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334003e20 con 0x7f934c1053b0 2024-04-17T07:55:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.087+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c1a2340 con 0x7f934c1053b0 2024-04-17T07:55:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.087+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c1a27e0 con 0x7f934c1053b0 2024-04-17T07:55:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.088+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c10b4e0 con 0x7f934c1053b0 2024-04-17T07:55:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.090+0000 7f9349600640 1 -- 172.21.15.46:0/2056851317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9334044070 con 0x7f934c1053b0 2024-04-17T07:55:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.091+0000 7f9349600640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93300a3bb0 0x7f93300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.092+0000 7f9349600640 1 -- 172.21.15.46:0/2056851317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f93340f21f0 con 0x7f934c1053b0 2024-04-17T07:55:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.092+0000 7f9351200640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93300a3bb0 0x7f93300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.093+0000 7f9351200640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93300a3bb0 0x7f93300a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f933c0095f0 tx=0x7f933c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.096+0000 7f9349600640 1 -- 172.21.15.46:0/2056851317 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f93340be2e0 con 0x7f934c1053b0 2024-04-17T07:55:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.317+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f934c02d5d0 con 0x7f934c1053b0 2024-04-17T07:55:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.318+0000 7f9349600640 1 -- 172.21.15.46:0/2056851317 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+585 (secure 0 0 0) 0x7f93340c25e0 con 0x7f934c1053b0 2024-04-17T07:55:46.318 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/461113542 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:46.318 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/461113542 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:46.318 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/663006341 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:46.318 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/63594551 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:46.319 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/61074341 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:46.319 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1435664148 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:46.319 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1911895676 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:46.319 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4094803685 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:46.319 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2289010020 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:46.319 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4094803685 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-04-17T07:55:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93300a3bb0 msgr2=0x7f93300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93300a3bb0 0x7f93300a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f933c0095f0 tx=0x7f933c005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c1053b0 msgr2=0x7f934c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c1053b0 0x7f934c19aa20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9334009230 tx=0x7f933402c650 comp rx=0 tx=0).stop 2024-04-17T07:55:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 shutdown_connections 2024-04-17T07:55:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93300a3bb0 0x7f93300a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f934c105d70 0x7f934c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f934c1053b0 0x7f934c19aa20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 --2- 172.21.15.46:0/2056851317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f934c1041c0 0x7f934c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 >> 172.21.15.46:0/2056851317 conn(0x7f934c0ff750 msgr2=0x7f934c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 shutdown_connections 2024-04-17T07:55:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.321+0000 7f9353c00640 1 -- 172.21.15.46:0/2056851317 wait complete. 2024-04-17T07:55:46.341 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:55:46.341 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:55:46.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 -- 172.21.15.46:0/673245921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2028105c70 msgr2=0x7f202810c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 --2- 172.21.15.46:0/673245921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2028105c70 0x7f202810c500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2020009be0 tx=0x7f202002bef0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 -- 172.21.15.46:0/673245921 shutdown_connections 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 --2- 172.21.15.46:0/673245921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2028105c70 0x7f202810c500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 --2- 172.21.15.46:0/673245921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20281052b0 0x7f2028105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 --2- 172.21.15.46:0/673245921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20281040c0 0x7f20281044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 -- 172.21.15.46:0/673245921 >> 172.21.15.46:0/673245921 conn(0x7f20280ff630 msgr2=0x7f2028101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 -- 172.21.15.46:0/673245921 shutdown_connections 2024-04-17T07:55:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.604+0000 7f202e200640 1 -- 172.21.15.46:0/673245921 wait complete. 2024-04-17T07:55:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 Processor -- start 2024-04-17T07:55:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 -- start start 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20281040c0 0x7f202819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20281052b0 0x7f202819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2028105c70 0x7f20281a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f202810fda0 con 0x7f2028105c70 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f202810fc20 con 0x7f20281040c0 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f202810ff20 con 0x7f20281052b0 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20281040c0 0x7f202819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f2027e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20281052b0 0x7f202819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202ce00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20281040c0 0x7f202819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:58192/0 (socket says 172.21.15.46:58192) 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.606+0000 7f202ce00640 1 -- 172.21.15.46:0/2739481067 learned_addr learned my addr 172.21.15.46:0/2739481067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202d800640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2028105c70 0x7f20281a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202ce00640 1 -- 172.21.15.46:0/2739481067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20281052b0 msgr2=0x7f202819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202ce00640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20281052b0 0x7f202819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202ce00640 1 -- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2028105c70 msgr2=0x7f20281a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202ce00640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2028105c70 0x7f20281a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202ce00640 1 -- 172.21.15.46:0/2739481067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20281a1f50 con 0x7f20281040c0 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202d800640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2028105c70 0x7f20281a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202ce00640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20281040c0 0x7f202819a330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f201800e9f0 tx=0x7f201800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f2025600640 1 -- 172.21.15.46:0/2739481067 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800ccc0 con 0x7f20281040c0 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f2025600640 1 -- 172.21.15.46:0/2739481067 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20180035f0 con 0x7f20281040c0 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f2025600640 1 -- 172.21.15.46:0/2739481067 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800ce60 con 0x7f20281040c0 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.607+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20281a2240 con 0x7f20281040c0 2024-04-17T07:55:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.608+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20281a2740 con 0x7f20281040c0 2024-04-17T07:55:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.609+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202810b380 con 0x7f20281040c0 2024-04-17T07:55:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.611+0000 7f2025600640 1 -- 172.21.15.46:0/2739481067 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20180037a0 con 0x7f20281040c0 2024-04-17T07:55:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.612+0000 7f2025600640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f200c0a3ba0 0x7f200c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.612+0000 7f2025600640 1 -- 172.21.15.46:0/2739481067 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f20180d0980 con 0x7f20281040c0 2024-04-17T07:55:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.612+0000 7f2027e00640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f200c0a3ba0 0x7f200c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.613+0000 7f2027e00640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f200c0a3ba0 0x7f200c0a6060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f202819b840 tx=0x7f201400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.617+0000 7f2025600640 1 -- 172.21.15.46:0/2739481067 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f201809d0c0 con 0x7f20281040c0 2024-04-17T07:55:46.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.819+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f202802d5d0 con 0x7f20281040c0 2024-04-17T07:55:46.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.821+0000 7f2025600640 1 -- 172.21.15.46:0/2739481067 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f201809c9c0 con 0x7f20281040c0 2024-04-17T07:55:46.822 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:46.822 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:46.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f200c0a3ba0 msgr2=0x7f200c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f200c0a3ba0 0x7f200c0a6060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f202819b840 tx=0x7f201400b390 comp rx=0 tx=0).stop 2024-04-17T07:55:46.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20281040c0 msgr2=0x7f202819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20281040c0 0x7f202819a330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f201800e9f0 tx=0x7f201800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 shutdown_connections 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f200c0a3ba0 0x7f200c0a6060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2028105c70 0x7f20281a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20281052b0 0x7f202819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 --2- 172.21.15.46:0/2739481067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20281040c0 0x7f202819a330 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 >> 172.21.15.46:0/2739481067 conn(0x7f20280ff630 msgr2=0x7f2028108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 shutdown_connections 2024-04-17T07:55:46.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:46.824+0000 7f202e200640 1 -- 172.21.15.46:0/2739481067 wait complete. 2024-04-17T07:55:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:55:46.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:55:46.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:46 smithi046 ceph-mon[32038]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-17T07:55:46.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:46 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4161262014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:55:46.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:46 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2056851317' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:55:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:46 smithi053 ceph-mon[36437]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-17T07:55:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:46 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4161262014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:55:46.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:46 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2056851317' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:55:47.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:46 smithi062 ceph-mon[36507]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-17T07:55:47.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:46 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4161262014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:55:47.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:46 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2056851317' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:55:47.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 -- 172.21.15.46:0/3852832057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 msgr2=0x7f1470109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 --2- 172.21.15.46:0/3852832057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f1470109a30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1458009900 tx=0x7f145802bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 -- 172.21.15.46:0/3852832057 shutdown_connections 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 --2- 172.21.15.46:0/3852832057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1470109f70 0x7f1470110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 --2- 172.21.15.46:0/3852832057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f1470109a30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 --2- 172.21.15.46:0/3852832057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14701083c0 0x7f14701087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 -- 172.21.15.46:0/3852832057 >> 172.21.15.46:0/3852832057 conn(0x7f14700773a0 msgr2=0x7f14700797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 -- 172.21.15.46:0/3852832057 shutdown_connections 2024-04-17T07:55:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.065+0000 7f1475e00640 1 -- 172.21.15.46:0/3852832057 wait complete. 2024-04-17T07:55:47.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.066+0000 7f1475e00640 1 Processor -- start 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.066+0000 7f1475e00640 1 -- start start 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.066+0000 7f1475e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14701083c0 0x7f147019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.066+0000 7f1475e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f147019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.066+0000 7f1475e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1470109f70 0x7f14701a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.066+0000 7f1475e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14701140a0 con 0x7f14701095b0 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1475e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1470113f20 con 0x7f14701083c0 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1475e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1470114220 con 0x7f1470109f70 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f146fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f147019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f146fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f147019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42728/0 (socket says 172.21.15.46:42728) 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f146fe00640 1 -- 172.21.15.46:0/3180807858 learned_addr learned my addr 172.21.15.46:0/3180807858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1474a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14701083c0 0x7f147019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1475400640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1470109f70 0x7f14701a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1474a00640 1 -- 172.21.15.46:0/3180807858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1470109f70 msgr2=0x7f14701a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1474a00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1470109f70 0x7f14701a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1474a00640 1 -- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 msgr2=0x7f147019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1474a00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f147019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f1474a00640 1 -- 172.21.15.46:0/3180807858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14701a6290 con 0x7f14701083c0 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.067+0000 7f146fe00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f147019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.068+0000 7f1474a00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14701083c0 0x7f147019e670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f146000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.068+0000 7f146d600640 1 -- 172.21.15.46:0/3180807858 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146000ccc0 con 0x7f14701083c0 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.068+0000 7f146d600640 1 -- 172.21.15.46:0/3180807858 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14600035f0 con 0x7f14701083c0 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.068+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14701a6580 con 0x7f14701083c0 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.068+0000 7f146d600640 1 -- 172.21.15.46:0/3180807858 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146000ce60 con 0x7f14701083c0 2024-04-17T07:55:47.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.068+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14701a6aa0 con 0x7f14701083c0 2024-04-17T07:55:47.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.069+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147010f680 con 0x7f14701083c0 2024-04-17T07:55:47.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.071+0000 7f146d600640 1 -- 172.21.15.46:0/3180807858 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14600037a0 con 0x7f14701083c0 2024-04-17T07:55:47.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.072+0000 7f146d600640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14540a3ba0 0x7f14540a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.072+0000 7f146d600640 1 -- 172.21.15.46:0/3180807858 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f14600d0a90 con 0x7f14701083c0 2024-04-17T07:55:47.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.073+0000 7f146fe00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14540a3ba0 0x7f14540a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.073+0000 7f146fe00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14540a3ba0 0x7f14540a6060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f145802c8e0 tx=0x7f145802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.077+0000 7f146d600640 1 -- 172.21.15.46:0/3180807858 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f146009d1d0 con 0x7f14701083c0 2024-04-17T07:55:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.281+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1470062b60 con 0x7f14701083c0 2024-04-17T07:55:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.282+0000 7f146d600640 1 -- 172.21.15.46:0/3180807858 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f146009cad0 con 0x7f14701083c0 2024-04-17T07:55:47.282 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:55:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14540a3ba0 msgr2=0x7f14540a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14540a3ba0 0x7f14540a6060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f145802c8e0 tx=0x7f145802df90 comp rx=0 tx=0).stop 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14701083c0 msgr2=0x7f147019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14701083c0 0x7f147019e670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f146000e9c0 tx=0x7f146000ee80 comp rx=0 tx=0).stop 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 shutdown_connections 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f14540a3ba0 0x7f14540a6060 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1470109f70 0x7f14701a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14701095b0 0x7f147019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 --2- 172.21.15.46:0/3180807858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14701083c0 0x7f147019e670 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 >> 172.21.15.46:0/3180807858 conn(0x7f14700773a0 msgr2=0x7f147010c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 shutdown_connections 2024-04-17T07:55:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.285+0000 7f1475e00640 1 -- 172.21.15.46:0/3180807858 wait complete. 2024-04-17T07:55:47.306 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:55:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.534+0000 7fcc5d600640 1 -- 172.21.15.46:0/4057290462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58101de0 msgr2=0x7fcc58102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.534+0000 7fcc5d600640 1 --2- 172.21.15.46:0/4057290462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58101de0 0x7fcc58102200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009900 tx=0x7fcc4802bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.534+0000 7fcc5d600640 1 -- 172.21.15.46:0/4057290462 shutdown_connections 2024-04-17T07:55:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.534+0000 7fcc5d600640 1 --2- 172.21.15.46:0/4057290462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc58103990 0x7fcc5810a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.534+0000 7fcc5d600640 1 --2- 172.21.15.46:0/4057290462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc58102fd0 0x7fcc58103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.535+0000 7fcc5d600640 1 --2- 172.21.15.46:0/4057290462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58101de0 0x7fcc58102200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.535+0000 7fcc5d600640 1 -- 172.21.15.46:0/4057290462 >> 172.21.15.46:0/4057290462 conn(0x7fcc580fd370 msgr2=0x7fcc580ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.535+0000 7fcc5d600640 1 -- 172.21.15.46:0/4057290462 shutdown_connections 2024-04-17T07:55:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.535+0000 7fcc5d600640 1 -- 172.21.15.46:0/4057290462 wait complete. 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 Processor -- start 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 -- start start 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc58101de0 0x7fcc581980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58102fd0 0x7fcc58198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc58103990 0x7fcc5819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5810da30 con 0x7fcc58102fd0 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5810d8b0 con 0x7fcc58101de0 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.536+0000 7fcc5d600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5810dbb0 con 0x7fcc58103990 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58102fd0 0x7fcc58198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58102fd0 0x7fcc58198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42744/0 (socket says 172.21.15.46:42744) 2024-04-17T07:55:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 -- 172.21.15.46:0/2920024609 learned_addr learned my addr 172.21.15.46:0/2920024609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc5cc00640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc58103990 0x7fcc5819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc57e00640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc58101de0 0x7fcc581980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 -- 172.21.15.46:0/2920024609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc58103990 msgr2=0x7fcc5819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc58103990 0x7fcc5819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 -- 172.21.15.46:0/2920024609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc58101de0 msgr2=0x7fcc581980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc58101de0 0x7fcc581980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 -- 172.21.15.46:0/2920024609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5819fd00 con 0x7fcc58102fd0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc57e00640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc58101de0 0x7fcc581980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc5cc00640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc58103990 0x7fcc5819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.537+0000 7fcc4f400640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58102fd0 0x7fcc58198620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000d950 tx=0x7fcc4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.538+0000 7fcc55600640 1 -- 172.21.15.46:0/2920024609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40017070 con 0x7fcc58102fd0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.538+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5819fff0 con 0x7fcc58102fd0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.538+0000 7fcc55600640 1 -- 172.21.15.46:0/2920024609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc40009c30 con 0x7fcc58102fd0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.538+0000 7fcc55600640 1 -- 172.21.15.46:0/2920024609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4000f6b0 con 0x7fcc58102fd0 2024-04-17T07:55:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.538+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581a0500 con 0x7fcc58102fd0 2024-04-17T07:55:47.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.539+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc58109220 con 0x7fcc58102fd0 2024-04-17T07:55:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.541+0000 7fcc55600640 1 -- 172.21.15.46:0/2920024609 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc400109d0 con 0x7fcc58102fd0 2024-04-17T07:55:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.542+0000 7fcc55600640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcc3c0a3c30 0x7fcc3c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.543+0000 7fcc55600640 1 -- 172.21.15.46:0/2920024609 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fcc400d02d0 con 0x7fcc58102fd0 2024-04-17T07:55:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.543+0000 7fcc57e00640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcc3c0a3c30 0x7fcc3c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.543+0000 7fcc57e00640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcc3c0a3c30 0x7fcc3c0a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcc4802c7b0 tx=0x7fcc48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.547+0000 7fcc55600640 1 -- 172.21.15.46:0/2920024609 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcc40009de0 con 0x7fcc58102fd0 2024-04-17T07:55:47.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.757+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcc58062c00 con 0x7fcc58102fd0 2024-04-17T07:55:47.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.758+0000 7fcc55600640 1 -- 172.21.15.46:0/2920024609 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fcc4009ca10 con 0x7fcc58102fd0 2024-04-17T07:55:47.758 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcc3c0a3c30 msgr2=0x7fcc3c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcc3c0a3c30 0x7fcc3c0a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcc4802c7b0 tx=0x7fcc48005fd0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58102fd0 msgr2=0x7fcc58198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58102fd0 0x7fcc58198620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000d950 tx=0x7fcc4000de10 comp rx=0 tx=0).stop 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 shutdown_connections 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcc3c0a3c30 0x7fcc3c0a60f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc58103990 0x7fcc5819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc58102fd0 0x7fcc58198620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 --2- 172.21.15.46:0/2920024609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc58101de0 0x7fcc581980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 >> 172.21.15.46:0/2920024609 conn(0x7fcc580fd370 msgr2=0x7fcc581063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:47.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 shutdown_connections 2024-04-17T07:55:47.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:47.761+0000 7fcc5d600640 1 -- 172.21.15.46:0/2920024609 wait complete. 2024-04-17T07:55:47.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:47 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2739481067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:47.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:47 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3180807858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 -- 172.21.15.46:0/738940059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec8075fa0 msgr2=0x7fbec80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 --2- 172.21.15.46:0/738940059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec8075fa0 0x7fbec80743f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009860 tx=0x7fbeb802bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 -- 172.21.15.46:0/738940059 shutdown_connections 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 --2- 172.21.15.46:0/738940059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec8109c60 0x7fbec810c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 --2- 172.21.15.46:0/738940059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec80749c0 0x7fbec8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 --2- 172.21.15.46:0/738940059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec8075fa0 0x7fbec80743f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 -- 172.21.15.46:0/738940059 >> 172.21.15.46:0/738940059 conn(0x7fbec80fd310 msgr2=0x7fbec80ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 -- 172.21.15.46:0/738940059 shutdown_connections 2024-04-17T07:55:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.004+0000 7fbecd600640 1 -- 172.21.15.46:0/738940059 wait complete. 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.005+0000 7fbecd600640 1 Processor -- start 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.005+0000 7fbecd600640 1 -- start start 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbecd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec80749c0 0x7fbec819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbecd600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec8075fa0 0x7fbec819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbecd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec8109c60 0x7fbec81a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbecd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810fbe0 con 0x7fbec8109c60 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbecd600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810fa60 con 0x7fbec8075fa0 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbecd600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810fd60 con 0x7fbec80749c0 2024-04-17T07:55:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbeccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec8109c60 0x7fbec81a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbec7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec80749c0 0x7fbec819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbec7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec8075fa0 0x7fbec819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbeccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec8109c60 0x7fbec81a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52090/0 (socket says 172.21.15.46:52090) 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.006+0000 7fbeccc00640 1 -- 172.21.15.46:0/3833338524 learned_addr learned my addr 172.21.15.46:0/3833338524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec7400640 1 -- 172.21.15.46:0/3833338524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec80749c0 msgr2=0x7fbec819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec7400640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec80749c0 0x7fbec819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec7400640 1 -- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec8109c60 msgr2=0x7fbec81a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec7400640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec8109c60 0x7fbec81a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec7400640 1 -- 172.21.15.46:0/3833338524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec81a1e70 con 0x7fbec8075fa0 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbeccc00640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec8109c60 0x7fbec81a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:48.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec7400640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec8075fa0 0x7fbec819a790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400e9f0 tx=0x7fbeb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:48.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec4c00640 1 -- 172.21.15.46:0/3833338524 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400ccc0 con 0x7fbec8075fa0 2024-04-17T07:55:48.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec81a2160 con 0x7fbec8075fa0 2024-04-17T07:55:48.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec4c00640 1 -- 172.21.15.46:0/3833338524 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb4010910 con 0x7fbec8075fa0 2024-04-17T07:55:48.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbec4c00640 1 -- 172.21.15.46:0/3833338524 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4018960 con 0x7fbec8075fa0 2024-04-17T07:55:48.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.007+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec81a2700 con 0x7fbec8075fa0 2024-04-17T07:55:48.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.009+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec8108990 con 0x7fbec8075fa0 2024-04-17T07:55:48.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.011+0000 7fbec4c00640 1 -- 172.21.15.46:0/3833338524 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbeb4010ac0 con 0x7fbec8075fa0 2024-04-17T07:55:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.013+0000 7fbec4c00640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbeac0a3ba0 0x7fbeac0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.013+0000 7fbec4c00640 1 -- 172.21.15.46:0/3833338524 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbeb40d0a40 con 0x7fbec8075fa0 2024-04-17T07:55:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.013+0000 7fbec7e00640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbeac0a3ba0 0x7fbeac0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.013+0000 7fbec7e00640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbeac0a3ba0 0x7fbeac0a6060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009830 tx=0x7fbeb8034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:48.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.014+0000 7fbec4c00640 1 -- 172.21.15.46:0/3833338524 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbeb409cb30 con 0x7fbec8075fa0 2024-04-17T07:55:48.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:47 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2739481067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:48.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:47 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3180807858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:48.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:47 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2739481067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:48.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:47 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3180807858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:55:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.224+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbec802d780 con 0x7fbec8075fa0 2024-04-17T07:55:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.225+0000 7fbec4c00640 1 -- 172.21.15.46:0/3833338524 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fbeb40a0e30 con 0x7fbec8075fa0 2024-04-17T07:55:48.225 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:55:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.227+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbeac0a3ba0 msgr2=0x7fbeac0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.227+0000 7fbecd600640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbeac0a3ba0 0x7fbeac0a6060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009830 tx=0x7fbeb8034ab0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.227+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec8075fa0 msgr2=0x7fbec819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.227+0000 7fbecd600640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec8075fa0 0x7fbec819a790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400e9f0 tx=0x7fbeb400eeb0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 shutdown_connections 2024-04-17T07:55:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fbeac0a3ba0 0x7fbeac0a6060 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbec8109c60 0x7fbec81a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbec8075fa0 0x7fbec819a790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 --2- 172.21.15.46:0/3833338524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbec80749c0 0x7fbec819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 >> 172.21.15.46:0/3833338524 conn(0x7fbec80fd310 msgr2=0x7fbec8105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 shutdown_connections 2024-04-17T07:55:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.228+0000 7fbecd600640 1 -- 172.21.15.46:0/3833338524 wait complete. 2024-04-17T07:55:48.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 -- 172.21.15.46:0/1994450146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341097b0 msgr2=0x7f4b34109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 --2- 172.21.15.46:0/1994450146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341097b0 0x7f4b34109c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009860 tx=0x7f4b2002bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 -- 172.21.15.46:0/1994450146 shutdown_connections 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 --2- 172.21.15.46:0/1994450146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 0x7f4b34110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 --2- 172.21.15.46:0/1994450146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341097b0 0x7f4b34109c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 --2- 172.21.15.46:0/1994450146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b341085c0 0x7f4b341089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 -- 172.21.15.46:0/1994450146 >> 172.21.15.46:0/1994450146 conn(0x7f4b34077570 msgr2=0x7f4b34079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 -- 172.21.15.46:0/1994450146 shutdown_connections 2024-04-17T07:55:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.474+0000 7f4b39a00640 1 -- 172.21.15.46:0/1994450146 wait complete. 2024-04-17T07:55:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.475+0000 7f4b39a00640 1 Processor -- start 2024-04-17T07:55:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.475+0000 7f4b39a00640 1 -- start start 2024-04-17T07:55:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b39a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341085c0 0x7f4b3419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b39a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b341097b0 0x7f4b3419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b39a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 0x7f4b341a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b39a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b34114230 con 0x7f4b341085c0 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b39a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b341140b0 con 0x7f4b341097b0 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b39a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4b341143b0 con 0x7f4b3410a170 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 0x7f4b341a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 0x7f4b341a5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36668/0 (socket says 172.21.15.46:36668) 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 -- 172.21.15.46:0/4087617228 learned_addr learned my addr 172.21.15.46:0/4087617228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b32000640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b341097b0 0x7f4b3419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b32a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341085c0 0x7f4b3419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 -- 172.21.15.46:0/4087617228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b341097b0 msgr2=0x7f4b3419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b341097b0 0x7f4b3419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 -- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341085c0 msgr2=0x7f4b3419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341085c0 0x7f4b3419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b33400640 1 -- 172.21.15.46:0/4087617228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b341a6550 con 0x7f4b3410a170 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b32a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341085c0 0x7f4b3419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.476+0000 7f4b32000640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b341097b0 0x7f4b3419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.477+0000 7f4b33400640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 0x7f4b341a5ee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00efc0 tx=0x7f4b2c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.477+0000 7f4b2b400640 1 -- 172.21.15.46:0/4087617228 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c003970 con 0x7f4b3410a170 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.477+0000 7f4b2b400640 1 -- 172.21.15.46:0/4087617228 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c004020 con 0x7f4b3410a170 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.477+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b341a6840 con 0x7f4b3410a170 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.477+0000 7f4b2b400640 1 -- 172.21.15.46:0/4087617228 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00b9f0 con 0x7f4b3410a170 2024-04-17T07:55:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.477+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b341a6d10 con 0x7f4b3410a170 2024-04-17T07:55:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.478+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3410f9b0 con 0x7f4b3410a170 2024-04-17T07:55:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.480+0000 7f4b2b400640 1 -- 172.21.15.46:0/4087617228 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4b2c01d020 con 0x7f4b3410a170 2024-04-17T07:55:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.482+0000 7f4b2b400640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4b180a3bb0 0x7f4b180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.482+0000 7f4b2b400640 1 -- 172.21.15.46:0/4087617228 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4b2c0d0ba0 con 0x7f4b3410a170 2024-04-17T07:55:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.482+0000 7f4b32a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4b180a3bb0 0x7f4b180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.483+0000 7f4b32a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4b180a3bb0 0x7f4b180a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b34109610 tx=0x7f4b24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.484+0000 7f4b2b400640 1 -- 172.21.15.46:0/4087617228 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4b2c0d0e20 con 0x7f4b3410a170 2024-04-17T07:55:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.691+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4b3402d760 con 0x7f4b3410a170 2024-04-17T07:55:48.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.692+0000 7f4b2b400640 1 -- 172.21.15.46:0/4087617228 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f4b2c09d2e0 con 0x7f4b3410a170 2024-04-17T07:55:48.693 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:55:48.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4b180a3bb0 msgr2=0x7f4b180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4b180a3bb0 0x7f4b180a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b34109610 tx=0x7f4b24005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:48.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 msgr2=0x7f4b341a5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 0x7f4b341a5ee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00efc0 tx=0x7f4b2c00c800 comp rx=0 tx=0).stop 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 shutdown_connections 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4b180a3bb0 0x7f4b180a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b3410a170 0x7f4b341a5ee0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b341097b0 0x7f4b3419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 --2- 172.21.15.46:0/4087617228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b341085c0 0x7f4b3419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 >> 172.21.15.46:0/4087617228 conn(0x7f4b34077570 msgr2=0x7f4b3410cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 shutdown_connections 2024-04-17T07:55:48.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.695+0000 7f4b39a00640 1 -- 172.21.15.46:0/4087617228 wait complete. 2024-04-17T07:55:48.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T07:55:48.922 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:48 smithi046 ceph-mon[32038]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:48.922 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:48 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2920024609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:55:48.922 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:48 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3833338524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:55:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 -- 172.21.15.46:0/519315266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 msgr2=0x7f8454110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 --2- 172.21.15.46:0/519315266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 0x7f8454110820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f844c009ca0 tx=0x7f844c02c100 comp rx=0 tx=0).stop 2024-04-17T07:55:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 -- 172.21.15.46:0/519315266 shutdown_connections 2024-04-17T07:55:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 --2- 172.21.15.46:0/519315266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 0x7f8454110820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 --2- 172.21.15.46:0/519315266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84541095d0 0x7f8454109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 --2- 172.21.15.46:0/519315266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541083e0 0x7f8454108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 -- 172.21.15.46:0/519315266 >> 172.21.15.46:0/519315266 conn(0x7f84540773a0 msgr2=0x7f84540797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 -- 172.21.15.46:0/519315266 shutdown_connections 2024-04-17T07:55:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.946+0000 7f8459000640 1 -- 172.21.15.46:0/519315266 wait complete. 2024-04-17T07:55:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.947+0000 7f8459000640 1 Processor -- start 2024-04-17T07:55:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.947+0000 7f8459000640 1 -- start start 2024-04-17T07:55:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.947+0000 7f8459000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84541083e0 0x7f845419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8459000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541095d0 0x7f845419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8459000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 0x7f84541a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8459000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8454114040 con 0x7f84541095d0 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8459000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8454113ec0 con 0x7f8454109f90 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8459000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84541141c0 con 0x7f84541083e0 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541095d0 0x7f845419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8453400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84541083e0 0x7f845419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541095d0 0x7f845419ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52138/0 (socket says 172.21.15.46:52138) 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 -- 172.21.15.46:0/2857605310 learned_addr learned my addr 172.21.15.46:0/2857605310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8453e00640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 0x7f84541a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 -- 172.21.15.46:0/2857605310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84541083e0 msgr2=0x7f845419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84541083e0 0x7f845419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 -- 172.21.15.46:0/2857605310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 msgr2=0x7f84541a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 0x7f84541a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8452a00640 1 -- 172.21.15.46:0/2857605310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84541a62d0 con 0x7f84541095d0 2024-04-17T07:55:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.948+0000 7f8453400640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84541083e0 0x7f845419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:48.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.949+0000 7f8452a00640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541095d0 0x7f845419ebf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9f0 tx=0x7f844000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:48.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.949+0000 7f844be00640 1 -- 172.21.15.46:0/2857605310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844000cd80 con 0x7f84541095d0 2024-04-17T07:55:48.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.949+0000 7f844be00640 1 -- 172.21.15.46:0/2857605310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8440010910 con 0x7f84541095d0 2024-04-17T07:55:48.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.949+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84541a65c0 con 0x7f84541095d0 2024-04-17T07:55:48.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.949+0000 7f844be00640 1 -- 172.21.15.46:0/2857605310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440018990 con 0x7f84541095d0 2024-04-17T07:55:48.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.949+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84541a6b60 con 0x7f84541095d0 2024-04-17T07:55:48.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.950+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845410f700 con 0x7f84541095d0 2024-04-17T07:55:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.953+0000 7f844be00640 1 -- 172.21.15.46:0/2857605310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8440010ac0 con 0x7f84541095d0 2024-04-17T07:55:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.954+0000 7f844be00640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f84380a3c30 0x7f84380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.955+0000 7f844be00640 1 -- 172.21.15.46:0/2857605310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f84400d08b0 con 0x7f84541095d0 2024-04-17T07:55:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.955+0000 7f8453400640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f84380a3c30 0x7f84380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:48.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.956+0000 7f8453400640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f84380a3c30 0x7f84380a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8454109430 tx=0x7f8444005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:48.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:48.960+0000 7f844be00640 1 -- 172.21.15.46:0/2857605310 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f844009ca00 con 0x7f84541095d0 2024-04-17T07:55:48.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:48 smithi053 ceph-mon[36437]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:48.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:48 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2920024609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:55:48.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:48 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3833338524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:55:49.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:48 smithi062 ceph-mon[36507]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:49.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:48 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2920024609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:55:49.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:48 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3833338524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.182+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f845402d5d0 con 0x7f84541095d0 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.183+0000 7f844be00640 1 -- 172.21.15.46:0/2857605310 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+585 (secure 0 0 0) 0x7f84400a0d00 con 0x7f84541095d0 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/461113542 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/461113542 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/663006341 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/63594551 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/61074341 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1435664148 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:49.184 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1911895676 2024-04-18T07:47:21.351556+0000 2024-04-17T07:55:49.185 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4094803685 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:49.185 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2289010020 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:49.185 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4094803685 2024-04-18T07:46:45.851840+0000 2024-04-17T07:55:49.185 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-04-17T07:55:49.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f84380a3c30 msgr2=0x7f84380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:49.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f84380a3c30 0x7f84380a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8454109430 tx=0x7f8444005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:49.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541095d0 msgr2=0x7f845419ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541095d0 0x7f845419ebf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9f0 tx=0x7f844000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 shutdown_connections 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f84380a3c30 0x7f84380a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8454109f90 0x7f84541a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84541095d0 0x7f845419ebf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 --2- 172.21.15.46:0/2857605310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84541083e0 0x7f845419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.186+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 >> 172.21.15.46:0/2857605310 conn(0x7f84540773a0 msgr2=0x7f845410c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.187+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 shutdown_connections 2024-04-17T07:55:49.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.187+0000 7f8459000640 1 -- 172.21.15.46:0/2857605310 wait complete. 2024-04-17T07:55:49.208 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-04-17T07:55:49.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:49.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:55:49.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[76003]: fuse finished with error 0 and tester_r 0 2024-04-17T07:55:49.310 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T07:55:49.395 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-17T07:55:49.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:55:49.395 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-17T07:55:49.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 -- 172.21.15.46:0/2003385216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 msgr2=0x7f77941044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 --2- 172.21.15.46:0/2003385216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 0x7f77941044c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7784009860 tx=0x7f778402bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 -- 172.21.15.46:0/2003385216 shutdown_connections 2024-04-17T07:55:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 --2- 172.21.15.46:0/2003385216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794105c50 0x7f779410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 --2- 172.21.15.46:0/2003385216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794105290 0x7f7794105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 --2- 172.21.15.46:0/2003385216 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 0x7f77941044c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 -- 172.21.15.46:0/2003385216 >> 172.21.15.46:0/2003385216 conn(0x7f77940ff630 msgr2=0x7f7794101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 -- 172.21.15.46:0/2003385216 shutdown_connections 2024-04-17T07:55:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.690+0000 7f7798a00640 1 -- 172.21.15.46:0/2003385216 wait complete. 2024-04-17T07:55:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 Processor -- start 2024-04-17T07:55:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 -- start start 2024-04-17T07:55:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 0x7f779419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794105290 0x7f779419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794105c50 0x7f77941a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f779410fd30 con 0x7f7794105290 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f779410fbb0 con 0x7f7794105c50 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.691+0000 7f7798a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f779410feb0 con 0x7f77941040a0 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7791e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794105290 0x7f779419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7792800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 0x7f779419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7791e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794105290 0x7f779419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52148/0 (socket says 172.21.15.46:52148) 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7791e00640 1 -- 172.21.15.46:0/2277176664 learned_addr learned my addr 172.21.15.46:0/2277176664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7793200640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794105c50 0x7f77941a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7792800640 1 -- 172.21.15.46:0/2277176664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794105c50 msgr2=0x7f77941a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7792800640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794105c50 0x7f77941a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7792800640 1 -- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794105290 msgr2=0x7f779419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7792800640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794105290 0x7f779419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7792800640 1 -- 172.21.15.46:0/2277176664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77941a1fd0 con 0x7f77941040a0 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7793200640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794105c50 0x7f77941a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.692+0000 7f7792800640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 0x7f779419a3b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7784009780 tx=0x7f7784032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.693+0000 7f778b400640 1 -- 172.21.15.46:0/2277176664 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784041020 con 0x7f77941040a0 2024-04-17T07:55:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.693+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77941a2260 con 0x7f77941040a0 2024-04-17T07:55:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.693+0000 7f778b400640 1 -- 172.21.15.46:0/2277176664 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778402c9f0 con 0x7f77941040a0 2024-04-17T07:55:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.693+0000 7f778b400640 1 -- 172.21.15.46:0/2277176664 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784039040 con 0x7f77941040a0 2024-04-17T07:55:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.693+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77941a2720 con 0x7f77941040a0 2024-04-17T07:55:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.694+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f779410b3c0 con 0x7f77941040a0 2024-04-17T07:55:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.696+0000 7f778b400640 1 -- 172.21.15.46:0/2277176664 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7784044020 con 0x7f77941040a0 2024-04-17T07:55:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.697+0000 7f778b400640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f77780a3ba0 0x7f77780a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.697+0000 7f778b400640 1 -- 172.21.15.46:0/2277176664 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f77840f1f40 con 0x7f77941040a0 2024-04-17T07:55:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.698+0000 7f7791e00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f77780a3ba0 0x7f77780a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.698+0000 7f7791e00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f77780a3ba0 0x7f77780a6060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f779419b8c0 tx=0x7f7780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:49.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.702+0000 7f778b400640 1 -- 172.21.15.46:0/2277176664 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f778404b020 con 0x7f77941040a0 2024-04-17T07:55:49.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:49 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4087617228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:55:49.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:49 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2857605310' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:55:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.912+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7794062b60 con 0x7f77941040a0 2024-04-17T07:55:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.914+0000 7f778b400640 1 -- 172.21.15.46:0/2277176664 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f77840be680 con 0x7f77941040a0 2024-04-17T07:55:49.915 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:49.915 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.916+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f77780a3ba0 msgr2=0x7f77780a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.916+0000 7f7798a00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f77780a3ba0 0x7f77780a6060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f779419b8c0 tx=0x7f7780009260 comp rx=0 tx=0).stop 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.916+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 msgr2=0x7f779419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.916+0000 7f7798a00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 0x7f779419a3b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7784009780 tx=0x7f7784032430 comp rx=0 tx=0).stop 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 shutdown_connections 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f77780a3ba0 0x7f77780a6060 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7794105c50 0x7f77941a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794105290 0x7f779419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 --2- 172.21.15.46:0/2277176664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77941040a0 0x7f779419a3b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:49.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 >> 172.21.15.46:0/2277176664 conn(0x7f77940ff630 msgr2=0x7f7794108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:49.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 shutdown_connections 2024-04-17T07:55:49.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:49.917+0000 7f7798a00640 1 -- 172.21.15.46:0/2277176664 wait complete. 2024-04-17T07:55:49.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:49 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4087617228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:55:49.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:49 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2857605310' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:55:50.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:49 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4087617228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:55:50.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:49 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2857605310' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:55:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 -- 172.21.15.46:0/489820535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce41052b0 msgr2=0x7f9ce4105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/489820535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce41052b0 0x7f9ce4105730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009900 tx=0x7f9ccc02bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 -- 172.21.15.46:0/489820535 shutdown_connections 2024-04-17T07:55:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/489820535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce4105c70 0x7f9ce410c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/489820535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce41052b0 0x7f9ce4105730 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/489820535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce41040c0 0x7f9ce41044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 -- 172.21.15.46:0/489820535 >> 172.21.15.46:0/489820535 conn(0x7f9ce40ff630 msgr2=0x7f9ce4101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 -- 172.21.15.46:0/489820535 shutdown_connections 2024-04-17T07:55:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.167+0000 7f9ce3e00640 1 -- 172.21.15.46:0/489820535 wait complete. 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 Processor -- start 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 -- start start 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce41040c0 0x7f9ce419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce41052b0 0x7f9ce419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce4105c70 0x7f9ce41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410fd50 con 0x7f9ce4105c70 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410fbd0 con 0x7f9ce41052b0 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.168+0000 7f9ce3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410fed0 con 0x7f9ce41040c0 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce41040c0 0x7f9ce419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce41040c0 0x7f9ce419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36734/0 (socket says 172.21.15.46:36734) 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce4105c70 0x7f9ce41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 -- 172.21.15.46:0/1358898920 learned_addr learned my addr 172.21.15.46:0/1358898920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:50.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9cda000640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce41052b0 0x7f9ce419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 -- 172.21.15.46:0/1358898920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce41052b0 msgr2=0x7f9ce419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce41052b0 0x7f9ce419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 -- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce4105c70 msgr2=0x7f9ce41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce4105c70 0x7f9ce41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 -- 172.21.15.46:0/1358898920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce41a1f80 con 0x7f9ce41040c0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.169+0000 7f9ce2a00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce41040c0 0x7f9ce419a360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9c0 tx=0x7f9cd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.170+0000 7f9cdbe00640 1 -- 172.21.15.46:0/1358898920 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400cde0 con 0x7f9ce41040c0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.170+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce41a2270 con 0x7f9ce41040c0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.170+0000 7f9cdbe00640 1 -- 172.21.15.46:0/1358898920 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cd4010910 con 0x7f9ce41040c0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.170+0000 7f9cdbe00640 1 -- 172.21.15.46:0/1358898920 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4018a10 con 0x7f9ce41040c0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.170+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce41a27e0 con 0x7f9ce41040c0 2024-04-17T07:55:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.171+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce410b3e0 con 0x7f9ce41040c0 2024-04-17T07:55:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.173+0000 7f9cdbe00640 1 -- 172.21.15.46:0/1358898920 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9cd401f020 con 0x7f9ce41040c0 2024-04-17T07:55:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.175+0000 7f9cdbe00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9cc80a3c20 0x7f9cc80a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.175+0000 7f9cdbe00640 1 -- 172.21.15.46:0/1358898920 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9cd40d07c0 con 0x7f9ce41040c0 2024-04-17T07:55:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.175+0000 7f9cda000640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9cc80a3c20 0x7f9cc80a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.176+0000 7f9cda000640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9cc80a3c20 0x7f9cc80a60e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc02c7c0 tx=0x7f9ccc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.179+0000 7f9cdbe00640 1 -- 172.21.15.46:0/1358898920 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9cd409cf00 con 0x7f9ce41040c0 2024-04-17T07:55:50.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.395+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ce4062b60 con 0x7f9ce41040c0 2024-04-17T07:55:50.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.397+0000 7f9cdbe00640 1 -- 172.21.15.46:0/1358898920 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f9cd409c800 con 0x7f9ce41040c0 2024-04-17T07:55:50.399 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:50.399 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9cc80a3c20 msgr2=0x7f9cc80a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9cc80a3c20 0x7f9cc80a60e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc02c7c0 tx=0x7f9ccc005fd0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce41040c0 msgr2=0x7f9ce419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce41040c0 0x7f9ce419a360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9c0 tx=0x7f9cd400ee80 comp rx=0 tx=0).stop 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 shutdown_connections 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9cc80a3c20 0x7f9cc80a60e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce4105c70 0x7f9ce41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce41052b0 0x7f9ce419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 --2- 172.21.15.46:0/1358898920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ce41040c0 0x7f9ce419a360 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 >> 172.21.15.46:0/1358898920 conn(0x7f9ce40ff630 msgr2=0x7f9ce41085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:50.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.400+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 shutdown_connections 2024-04-17T07:55:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.401+0000 7f9ce3e00640 1 -- 172.21.15.46:0/1358898920 wait complete. 2024-04-17T07:55:50.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T07:55:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.653+0000 7fd09f200640 1 -- 172.21.15.46:0/2369963546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098102850 msgr2=0x7fd098104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.653+0000 7fd09f200640 1 --2- 172.21.15.46:0/2369963546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098102850 0x7fd098104c40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009900 tx=0x7fd08c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:55:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.654+0000 7fd09f200640 1 -- 172.21.15.46:0/2369963546 shutdown_connections 2024-04-17T07:55:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.654+0000 7fd09f200640 1 --2- 172.21.15.46:0/2369963546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098107ab0 0x7fd098109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.654+0000 7fd09f200640 1 --2- 172.21.15.46:0/2369963546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098105180 0x7fd098107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.654+0000 7fd09f200640 1 --2- 172.21.15.46:0/2369963546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098102850 0x7fd098104c40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.654+0000 7fd09f200640 1 -- 172.21.15.46:0/2369963546 >> 172.21.15.46:0/2369963546 conn(0x7fd0980fc240 msgr2=0x7fd0980fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:50.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.654+0000 7fd09f200640 1 -- 172.21.15.46:0/2369963546 shutdown_connections 2024-04-17T07:55:50.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.654+0000 7fd09f200640 1 -- 172.21.15.46:0/2369963546 wait complete. 2024-04-17T07:55:50.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 Processor -- start 2024-04-17T07:55:50.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 -- start start 2024-04-17T07:55:50.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098102850 0x7fd0981021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098105180 0x7fd098100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098107ab0 0x7fd098100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd09810dad0 con 0x7fd098107ab0 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd09810d950 con 0x7fd098105180 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.655+0000 7fd09f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd09810dc50 con 0x7fd098102850 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098102850 0x7fd0981021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09d400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098105180 0x7fd098100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098102850 0x7fd0981021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36760/0 (socket says 172.21.15.46:36760) 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 -- 172.21.15.46:0/2768819437 learned_addr learned my addr 172.21.15.46:0/2768819437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09d400640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098105180 0x7fd098100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42302/0 (socket says 172.21.15.46:42302) 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09e800640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098107ab0 0x7fd098100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 -- 172.21.15.46:0/2768819437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098105180 msgr2=0x7fd098100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098105180 0x7fd098100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 -- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098107ab0 msgr2=0x7fd098100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098107ab0 0x7fd098100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 -- 172.21.15.46:0/2768819437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0981015d0 con 0x7fd098102850 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09e800640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098107ab0 0x7fd098100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09d400640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098105180 0x7fd098100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.656+0000 7fd09de00640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098102850 0x7fd0981021c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd08c02c260 tx=0x7fd08c02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.657+0000 7fd092a00640 1 -- 172.21.15.46:0/2768819437 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c041070 con 0x7fd098102850 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.657+0000 7fd092a00640 1 -- 172.21.15.46:0/2768819437 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c004450 con 0x7fd098102850 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.657+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0981a2270 con 0x7fd098102850 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.657+0000 7fd092a00640 1 -- 172.21.15.46:0/2768819437 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c034440 con 0x7fd098102850 2024-04-17T07:55:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.657+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a27c0 con 0x7fd098102850 2024-04-17T07:55:50.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.658+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0980fdd30 con 0x7fd098102850 2024-04-17T07:55:50.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.660+0000 7fd092a00640 1 -- 172.21.15.46:0/2768819437 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd08c044070 con 0x7fd098102850 2024-04-17T07:55:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.661+0000 7fd092a00640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd0800a3bb0 0x7fd0800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.661+0000 7fd092a00640 1 -- 172.21.15.46:0/2768819437 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd08c0f2600 con 0x7fd098102850 2024-04-17T07:55:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.662+0000 7fd09d400640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd0800a3bb0 0x7fd0800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:50.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.662+0000 7fd09d400640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd0800a3bb0 0x7fd0800a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd098101f30 tx=0x7fd088005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:50.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.664+0000 7fd092a00640 1 -- 172.21.15.46:0/2768819437 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd08c0bed40 con 0x7fd098102850 2024-04-17T07:55:50.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:50 smithi046 ceph-mon[32038]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:50.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:50 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2277176664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:50.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:50 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1358898920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:50.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:50.914+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd09802d5d0 con 0x7fd098102850 2024-04-17T07:55:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:50 smithi053 ceph-mon[36437]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:50 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2277176664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:50.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:50 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1358898920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:51.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:50 smithi062 ceph-mon[36507]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:51.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:50 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2277176664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:51.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:50 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1358898920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:51.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.681+0000 7fd092a00640 1 -- 172.21.15.46:0/2768819437 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fd08c0ba020 con 0x7fd098102850 2024-04-17T07:55:51.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.684+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd0800a3bb0 msgr2=0x7fd0800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:51.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.684+0000 7fd09f200640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd0800a3bb0 0x7fd0800a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd098101f30 tx=0x7fd088005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.684+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098102850 msgr2=0x7fd0981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.684+0000 7fd09f200640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098102850 0x7fd0981021c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd08c02c260 tx=0x7fd08c02c670 comp rx=0 tx=0).stop 2024-04-17T07:55:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 shutdown_connections 2024-04-17T07:55:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd0800a3bb0 0x7fd0800a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd098107ab0 0x7fd098100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd098105180 0x7fd098100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 --2- 172.21.15.46:0/2768819437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd098102850 0x7fd0981021c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 >> 172.21.15.46:0/2768819437 conn(0x7fd0980fc240 msgr2=0x7fd0981030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 shutdown_connections 2024-04-17T07:55:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.685+0000 7fd09f200640 1 -- 172.21.15.46:0/2768819437 wait complete. 2024-04-17T07:55:51.707 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-17T07:55:51.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-17T07:55:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.947+0000 7f8893200640 1 -- 172.21.15.46:0/2586572280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 msgr2=0x7f8894109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.947+0000 7f8893200640 1 --2- 172.21.15.46:0/2586572280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 0x7f8894109a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f887c009900 tx=0x7f887c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:55:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.948+0000 7f8893200640 1 -- 172.21.15.46:0/2586572280 shutdown_connections 2024-04-17T07:55:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.948+0000 7f8893200640 1 --2- 172.21.15.46:0/2586572280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8894109f90 0x7f8894110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.948+0000 7f8893200640 1 --2- 172.21.15.46:0/2586572280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 0x7f8894109a50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.948+0000 7f8893200640 1 --2- 172.21.15.46:0/2586572280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88941083e0 0x7f8894108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.948+0000 7f8893200640 1 -- 172.21.15.46:0/2586572280 >> 172.21.15.46:0/2586572280 conn(0x7f88940773a0 msgr2=0x7f88940797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.948+0000 7f8893200640 1 -- 172.21.15.46:0/2586572280 shutdown_connections 2024-04-17T07:55:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.948+0000 7f8893200640 1 -- 172.21.15.46:0/2586572280 wait complete. 2024-04-17T07:55:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 Processor -- start 2024-04-17T07:55:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 -- start start 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88941083e0 0x7f889419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 0x7f889419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894109f90 0x7f88941a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8894114070 con 0x7f88941095d0 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8894113ef0 con 0x7f8894109f90 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.949+0000 7f8893200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88941141f0 con 0x7f88941083e0 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88941083e0 0x7f889419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 0x7f889419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88941083e0 0x7f889419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36776/0 (socket says 172.21.15.46:36776) 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 -- 172.21.15.46:0/3743763518 learned_addr learned my addr 172.21.15.46:0/3743763518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8892800640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894109f90 0x7f88941a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 -- 172.21.15.46:0/3743763518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894109f90 msgr2=0x7f88941a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894109f90 0x7f88941a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 -- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 msgr2=0x7f889419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 0x7f889419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 -- 172.21.15.46:0/3743763518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88941a6350 con 0x7f88941083e0 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891400640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 0x7f889419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.950+0000 7f8891e00640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88941083e0 0x7f889419e7d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9c0 tx=0x7f888400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.951+0000 7f888aa00640 1 -- 172.21.15.46:0/3743763518 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888400cd80 con 0x7f88941083e0 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.951+0000 7f888aa00640 1 -- 172.21.15.46:0/3743763518 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88840035f0 con 0x7f88941083e0 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.951+0000 7f888aa00640 1 -- 172.21.15.46:0/3743763518 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884010690 con 0x7f88941083e0 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.951+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88941a6640 con 0x7f88941083e0 2024-04-17T07:55:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.951+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88941a6b10 con 0x7f88941083e0 2024-04-17T07:55:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.952+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889410f780 con 0x7f88941083e0 2024-04-17T07:55:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.954+0000 7f888aa00640 1 -- 172.21.15.46:0/3743763518 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f88840037a0 con 0x7f88941083e0 2024-04-17T07:55:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.956+0000 7f888aa00640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88780a3bb0 0x7f88780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.956+0000 7f888aa00640 1 -- 172.21.15.46:0/3743763518 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f88840d0aa0 con 0x7f88941083e0 2024-04-17T07:55:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.956+0000 7f8891400640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88780a3bb0 0x7f88780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:51.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.957+0000 7f8891400640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88780a3bb0 0x7f88780a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f887c02c7b0 tx=0x7f887c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:51.960+0000 7f888aa00640 1 -- 172.21.15.46:0/3743763518 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f888409d1e0 con 0x7f88941083e0 2024-04-17T07:55:51.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:51 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2768819437' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:55:51.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:51 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:55:52.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:51 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2768819437' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:55:52.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:51 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:55:52.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:51 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2768819437' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:55:52.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:51 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:55:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.211+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f8894062b90 con 0x7f88941083e0 2024-04-17T07:55:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.695+0000 7f888aa00640 1 -- 172.21.15.46:0/3743763518 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f888409cae0 con 0x7f88941083e0 2024-04-17T07:55:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88780a3bb0 msgr2=0x7f88780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88780a3bb0 0x7f88780a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f887c02c7b0 tx=0x7f887c005fd0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88941083e0 msgr2=0x7f889419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88941083e0 0x7f889419e7d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9c0 tx=0x7f888400ee80 comp rx=0 tx=0).stop 2024-04-17T07:55:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 shutdown_connections 2024-04-17T07:55:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88780a3bb0 0x7f88780a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894109f90 0x7f88941a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88941095d0 0x7f889419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 --2- 172.21.15.46:0/3743763518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88941083e0 0x7f889419e7d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.699+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 >> 172.21.15.46:0/3743763518 conn(0x7f88940773a0 msgr2=0x7f889410c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.700+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 shutdown_connections 2024-04-17T07:55:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.700+0000 7f8893200640 1 -- 172.21.15.46:0/3743763518 wait complete. 2024-04-17T07:55:52.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 -- 172.21.15.46:0/1084484119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4102850 msgr2=0x7f54a4104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1084484119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4102850 0x7f54a4104c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5494009be0 tx=0x7f549402bef0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 -- 172.21.15.46:0/1084484119 shutdown_connections 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1084484119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4107ab0 0x7f54a4109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1084484119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4105180 0x7f54a4107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1084484119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4102850 0x7f54a4104c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 -- 172.21.15.46:0/1084484119 >> 172.21.15.46:0/1084484119 conn(0x7f54a40fc240 msgr2=0x7f54a40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 -- 172.21.15.46:0/1084484119 shutdown_connections 2024-04-17T07:55:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.933+0000 7f54a9a00640 1 -- 172.21.15.46:0/1084484119 wait complete. 2024-04-17T07:55:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.934+0000 7f54a9a00640 1 Processor -- start 2024-04-17T07:55:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.935+0000 7f54a9a00640 1 -- start start 2024-04-17T07:55:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.935+0000 7f54a9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4102850 0x7f54a41021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.935+0000 7f54a9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4105180 0x7f54a4100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.935+0000 7f54a9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4107ab0 0x7f54a4100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.935+0000 7f54a9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410dac0 con 0x7f54a4107ab0 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.935+0000 7f54a9a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410d940 con 0x7f54a4105180 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.935+0000 7f54a9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410dc40 con 0x7f54a4102850 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4107ab0 0x7f54a4100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4102850 0x7f54a41021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4107ab0 0x7f54a4100d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52206/0 (socket says 172.21.15.46:52206) 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a9000640 1 -- 172.21.15.46:0/1328776605 learned_addr learned my addr 172.21.15.46:0/1328776605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4102850 0x7f54a41021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36788/0 (socket says 172.21.15.46:36788) 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f549b400640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4105180 0x7f54a4100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 -- 172.21.15.46:0/1328776605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4105180 msgr2=0x7f54a4100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4105180 0x7f54a4100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 -- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4107ab0 msgr2=0x7f54a4100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4107ab0 0x7f54a4100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 -- 172.21.15.46:0/1328776605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a41015d0 con 0x7f54a4102850 2024-04-17T07:55:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.936+0000 7f54a3e00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4102850 0x7f54a41021c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f549402c500 tx=0x7f5494033580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.937+0000 7f54a1600640 1 -- 172.21.15.46:0/1328776605 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494043020 con 0x7f54a4102850 2024-04-17T07:55:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.937+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a41a2200 con 0x7f54a4102850 2024-04-17T07:55:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.937+0000 7f54a1600640 1 -- 172.21.15.46:0/1328776605 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494003e50 con 0x7f54a4102850 2024-04-17T07:55:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.937+0000 7f54a1600640 1 -- 172.21.15.46:0/1328776605 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494034690 con 0x7f54a4102850 2024-04-17T07:55:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.937+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a41a27a0 con 0x7f54a4102850 2024-04-17T07:55:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.938+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a40fdd30 con 0x7f54a4102850 2024-04-17T07:55:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.940+0000 7f54a1600640 1 -- 172.21.15.46:0/1328776605 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5494046020 con 0x7f54a4102850 2024-04-17T07:55:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.942+0000 7f54a1600640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f54880a3c30 0x7f54880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.942+0000 7f54a1600640 1 -- 172.21.15.46:0/1328776605 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f54940f2c30 con 0x7f54a4102850 2024-04-17T07:55:52.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.942+0000 7f549b400640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f54880a3c30 0x7f54880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:52.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.943+0000 7f549b400640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f54880a3c30 0x7f54880a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f54a4101f30 tx=0x7f548c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:52.947+0000 7f54a1600640 1 -- 172.21.15.46:0/1328776605 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f549404d020 con 0x7f54a4102850 2024-04-17T07:55:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:52 smithi053 ceph-mon[36437]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:52 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T07:55:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:52 smithi053 ceph-mon[36437]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:52 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3743763518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T07:55:52.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:52 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T07:55:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:52 smithi062 ceph-mon[36507]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:52 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T07:55:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:52 smithi062 ceph-mon[36507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:52 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3743763518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T07:55:53.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:52 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T07:55:53.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:52 smithi046 ceph-mon[32038]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:55:53.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:52 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T07:55:53.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:52 smithi046 ceph-mon[32038]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:53.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:52 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3743763518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T07:55:53.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:52 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-17T07:55:53.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.199+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f54a402d5d0 con 0x7f54a4102850 2024-04-17T07:55:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.708+0000 7f54a1600640 1 -- 172.21.15.46:0/1328776605 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f54940bf370 con 0x7f54a4102850 2024-04-17T07:55:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f54880a3c30 msgr2=0x7f54880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f54880a3c30 0x7f54880a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f54a4101f30 tx=0x7f548c009260 comp rx=0 tx=0).stop 2024-04-17T07:55:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4102850 msgr2=0x7f54a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4102850 0x7f54a41021c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f549402c500 tx=0x7f5494033580 comp rx=0 tx=0).stop 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 shutdown_connections 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f54880a3c30 0x7f54880a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4107ab0 0x7f54a4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4105180 0x7f54a4100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 --2- 172.21.15.46:0/1328776605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54a4102850 0x7f54a41021c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.712+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 >> 172.21.15.46:0/1328776605 conn(0x7f54a40fc240 msgr2=0x7f54a41030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.713+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 shutdown_connections 2024-04-17T07:55:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.713+0000 7f54a9a00640 1 -- 172.21.15.46:0/1328776605 wait complete. 2024-04-17T07:55:53.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-17T07:55:53.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 -- 172.21.15.46:0/1587785793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec780749c0 msgr2=0x7fec78074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 --2- 172.21.15.46:0/1587785793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec780749c0 0x7fec78074e40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec64009900 tx=0x7fec6402bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 -- 172.21.15.46:0/1587785793 shutdown_connections 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 --2- 172.21.15.46:0/1587785793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec78109c90 0x7fec7810c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 --2- 172.21.15.46:0/1587785793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec780749c0 0x7fec78074e40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 --2- 172.21.15.46:0/1587785793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec78075fa0 0x7fec780743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 -- 172.21.15.46:0/1587785793 >> 172.21.15.46:0/1587785793 conn(0x7fec780fd340 msgr2=0x7fec780ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 -- 172.21.15.46:0/1587785793 shutdown_connections 2024-04-17T07:55:53.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.938+0000 7fec7f000640 1 -- 172.21.15.46:0/1587785793 wait complete. 2024-04-17T07:55:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.939+0000 7fec7f000640 1 Processor -- start 2024-04-17T07:55:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.939+0000 7fec7f000640 1 -- start start 2024-04-17T07:55:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.939+0000 7fec7f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec780749c0 0x7fec7819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7f000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec78075fa0 0x7fec7819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7f000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec78109c90 0x7fec781a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fc60 con 0x7fec780749c0 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7f000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fae0 con 0x7fec78109c90 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7f000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810fde0 con 0x7fec78075fa0 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec780749c0 0x7fec7819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec78109c90 0x7fec781a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec78075fa0 0x7fec7819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec78109c90 0x7fec781a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42344/0 (socket says 172.21.15.46:42344) 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 -- 172.21.15.46:0/2497157127 learned_addr learned my addr 172.21.15.46:0/2497157127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 -- 172.21.15.46:0/2497157127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec78075fa0 msgr2=0x7fec7819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec78075fa0 0x7fec7819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 -- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec780749c0 msgr2=0x7fec7819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:53.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec780749c0 0x7fec7819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.940+0000 7fec7e600640 1 -- 172.21.15.46:0/2497157127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec781a1f10 con 0x7fec78109c90 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.941+0000 7fec7e600640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec78109c90 0x7fec781a18a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec70009fd0 tx=0x7fec7000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.941+0000 7fec76000640 1 -- 172.21.15.46:0/2497157127 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70017070 con 0x7fec78109c90 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.941+0000 7fec76000640 1 -- 172.21.15.46:0/2497157127 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec70003e50 con 0x7fec78109c90 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.941+0000 7fec76000640 1 -- 172.21.15.46:0/2497157127 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec700106c0 con 0x7fec78109c90 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.941+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec781a2200 con 0x7fec78109c90 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.941+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec781a26d0 con 0x7fec78109c90 2024-04-17T07:55:53.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.942+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec781089c0 con 0x7fec78109c90 2024-04-17T07:55:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.944+0000 7fec76000640 1 -- 172.21.15.46:0/2497157127 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fec70003970 con 0x7fec78109c90 2024-04-17T07:55:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.945+0000 7fec76000640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fec600a3c30 0x7fec600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.946+0000 7fec76000640 1 -- 172.21.15.46:0/2497157127 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fec700d0310 con 0x7fec78109c90 2024-04-17T07:55:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.946+0000 7fec7dc00640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fec600a3c30 0x7fec600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:53.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.946+0000 7fec7dc00640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fec600a3c30 0x7fec600a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00bbe0 tx=0x7fec6c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:53.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:53.950+0000 7fec76000640 1 -- 172.21.15.46:0/2497157127 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fec7009c400 con 0x7fec78109c90 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1328776605' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:55:53.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:53 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1328776605' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:55:54.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:53 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1328776605' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:55:54.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:53 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:55:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.199+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fec78062b60 con 0x7fec78109c90 2024-04-17T07:55:54.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.525+0000 7fec76000640 1 -- 172.21.15.46:0/2497157127 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fec700a0700 con 0x7fec78109c90 2024-04-17T07:55:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fec600a3c30 msgr2=0x7fec600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fec600a3c30 0x7fec600a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00bbe0 tx=0x7fec6c00b390 comp rx=0 tx=0).stop 2024-04-17T07:55:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec78109c90 msgr2=0x7fec781a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec78109c90 0x7fec781a18a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec70009fd0 tx=0x7fec7000b260 comp rx=0 tx=0).stop 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 shutdown_connections 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fec600a3c30 0x7fec600a60f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec78109c90 0x7fec781a18a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec78075fa0 0x7fec7819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 --2- 172.21.15.46:0/2497157127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec780749c0 0x7fec7819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 >> 172.21.15.46:0/2497157127 conn(0x7fec780fd340 msgr2=0x7fec78105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 shutdown_connections 2024-04-17T07:55:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.529+0000 7fec7f000640 1 -- 172.21.15.46:0/2497157127 wait complete. 2024-04-17T07:55:54.550 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-17T07:55:54.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 -- 172.21.15.46:0/1459241177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 msgr2=0x7fc9fc105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 --2- 172.21.15.46:0/1459241177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 0x7fc9fc105de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009900 tx=0x7fc9f402bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 -- 172.21.15.46:0/1459241177 shutdown_connections 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 --2- 172.21.15.46:0/1459241177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 0x7fc9fc105de0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 --2- 172.21.15.46:0/1459241177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc102fb0 0x7fc9fc105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 --2- 172.21.15.46:0/1459241177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9fc100640 0x7fc9fc102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 -- 172.21.15.46:0/1459241177 >> 172.21.15.46:0/1459241177 conn(0x7fc9fc0fc240 msgr2=0x7fc9fc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 -- 172.21.15.46:0/1459241177 shutdown_connections 2024-04-17T07:55:54.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.781+0000 7fca02400640 1 -- 172.21.15.46:0/1459241177 wait complete. 2024-04-17T07:55:54.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.782+0000 7fca02400640 1 Processor -- start 2024-04-17T07:55:54.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.782+0000 7fca02400640 1 -- start start 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca02400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9fc100640 0x7fc9fc19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca02400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc102fb0 0x7fc9fc19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca02400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 0x7fc9fc1a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca02400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc1140c0 con 0x7fc9fc100640 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca02400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc113f40 con 0x7fc9fc105960 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca02400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc9fc114240 con 0x7fc9fc102fb0 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9fc100640 0x7fc9fc19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9fc100640 0x7fc9fc19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52234/0 (socket says 172.21.15.46:52234) 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01000640 1 -- 172.21.15.46:0/2988800791 learned_addr learned my addr 172.21.15.46:0/2988800791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 0x7fc9fc1a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fc9fbe00640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc102fb0 0x7fc9fc19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01a00640 1 -- 172.21.15.46:0/2988800791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc102fb0 msgr2=0x7fc9fc19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:54.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01a00640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc102fb0 0x7fc9fc19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01a00640 1 -- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9fc100640 msgr2=0x7fc9fc19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01a00640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9fc100640 0x7fc9fc19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.783+0000 7fca01a00640 1 -- 172.21.15.46:0/2988800791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9fc1a6280 con 0x7fc9fc105960 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.784+0000 7fca01a00640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 0x7fc9fc1a5c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc9f402c180 tx=0x7fc9f402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.784+0000 7fc9f9600640 1 -- 172.21.15.46:0/2988800791 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4041070 con 0x7fc9fc105960 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.784+0000 7fc9f9600640 1 -- 172.21.15.46:0/2988800791 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f4004450 con 0x7fc9fc105960 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.784+0000 7fc9f9600640 1 -- 172.21.15.46:0/2988800791 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4003e00 con 0x7fc9fc105960 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.784+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9fc1a6510 con 0x7fc9fc105960 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.784+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9fc1a6a20 con 0x7fc9fc105960 2024-04-17T07:55:54.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.786+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9fc0fdd30 con 0x7fc9fc105960 2024-04-17T07:55:54.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.787+0000 7fc9f9600640 1 -- 172.21.15.46:0/2988800791 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc9f4044070 con 0x7fc9fc105960 2024-04-17T07:55:54.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.789+0000 7fc9f9600640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc9e40a3c30 0x7fc9e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:54.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.789+0000 7fc9f9600640 1 -- 172.21.15.46:0/2988800791 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc9f40f2090 con 0x7fc9fc105960 2024-04-17T07:55:54.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.789+0000 7fca01000640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc9e40a3c30 0x7fc9e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:54.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.790+0000 7fca01000640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc9e40a3c30 0x7fc9e40a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009fd0 tx=0x7fc9f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:54.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:54.793+0000 7fc9f9600640 1 -- 172.21.15.46:0/2988800791 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc9f40be210 con 0x7fc9fc105960 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] up:active 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: Cluster is now healthy 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-17T07:55:54.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-04-17T07:55:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-17T07:55:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2497157127' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:55:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:55:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T07:55:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] up:active 2024-04-17T07:55:54.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:54 smithi046 ceph-mon[32038]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T07:55:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-17T07:55:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T07:55:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] up:active 2024-04-17T07:55:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T07:55:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T07:55:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: Cluster is now healthy 2024-04-17T07:55:54.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-17T07:55:54.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-04-17T07:55:54.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-17T07:55:54.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2497157127' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:55:54.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:55:54.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T07:55:54.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] up:active 2024-04-17T07:55:54.972 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:54 smithi053 ceph-mon[36437]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T07:55:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.028+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc9fc0623a0 con 0x7fc9e40a3c30 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] up:active 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: Cluster is now healthy 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2497157127' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:55:55.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T07:55:55.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] up:active 2024-04-17T07:55:55.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:54 smithi062 ceph-mon[36507]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T07:55:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.064+0000 7fc9f9600640 1 -- 172.21.15.46:0/2988800791 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9fc0623a0 con 0x7fc9e40a3c30 2024-04-17T07:55:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc9e40a3c30 msgr2=0x7fc9e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc9e40a3c30 0x7fc9e40a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009fd0 tx=0x7fc9f0009500 comp rx=0 tx=0).stop 2024-04-17T07:55:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 msgr2=0x7fc9fc1a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 0x7fc9fc1a5c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc9f402c180 tx=0x7fc9f402c650 comp rx=0 tx=0).stop 2024-04-17T07:55:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 shutdown_connections 2024-04-17T07:55:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc9e40a3c30 0x7fc9e40a60f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9fc105960 0x7fc9fc1a5c10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9fc102fb0 0x7fc9fc19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 --2- 172.21.15.46:0/2988800791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9fc100640 0x7fc9fc19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 >> 172.21.15.46:0/2988800791 conn(0x7fc9fc0fc240 msgr2=0x7fc9fc103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 shutdown_connections 2024-04-17T07:55:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.067+0000 7fca02400640 1 -- 172.21.15.46:0/2988800791 wait complete. 2024-04-17T07:55:55.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-17T07:55:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.287+0000 7fc9ab800640 1 -- 172.21.15.46:0/2221924316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a41041b0 msgr2=0x7fc9a41045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.287+0000 7fc9ab800640 1 --2- 172.21.15.46:0/2221924316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a41041b0 0x7fc9a41045d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc998009900 tx=0x7fc99802bc50 comp rx=0 tx=0).stop 2024-04-17T07:55:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.288+0000 7fc9ab800640 1 -- 172.21.15.46:0/2221924316 shutdown_connections 2024-04-17T07:55:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.288+0000 7fc9ab800640 1 --2- 172.21.15.46:0/2221924316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a4105d60 0x7fc9a410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.288+0000 7fc9ab800640 1 --2- 172.21.15.46:0/2221924316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9a41053a0 0x7fc9a4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.288+0000 7fc9ab800640 1 --2- 172.21.15.46:0/2221924316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a41041b0 0x7fc9a41045d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.288+0000 7fc9ab800640 1 -- 172.21.15.46:0/2221924316 >> 172.21.15.46:0/2221924316 conn(0x7fc9a40ff720 msgr2=0x7fc9a4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.288+0000 7fc9ab800640 1 -- 172.21.15.46:0/2221924316 shutdown_connections 2024-04-17T07:55:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.288+0000 7fc9ab800640 1 -- 172.21.15.46:0/2221924316 wait complete. 2024-04-17T07:55:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 Processor -- start 2024-04-17T07:55:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 -- start start 2024-04-17T07:55:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a41041b0 0x7fc9a419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9a41053a0 0x7fc9a419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a4105d60 0x7fc9a41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a410fe90 con 0x7fc9a4105d60 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a410fd10 con 0x7fc9a41053a0 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9ab800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4110010 con 0x7fc9a41041b0 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9a9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a4105d60 0x7fc9a41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9a8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a41041b0 0x7fc9a419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9a9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a4105d60 0x7fc9a41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52256/0 (socket says 172.21.15.46:52256) 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9a8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a41041b0 0x7fc9a419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36856/0 (socket says 172.21.15.46:36856) 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9a9800640 1 -- 172.21.15.46:0/896236962 learned_addr learned my addr 172.21.15.46:0/896236962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.289+0000 7fc9a3e00640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9a41053a0 0x7fc9a419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a8e00640 1 -- 172.21.15.46:0/896236962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9a41053a0 msgr2=0x7fc9a419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a8e00640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9a41053a0 0x7fc9a419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a8e00640 1 -- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a4105d60 msgr2=0x7fc9a41a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a8e00640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a4105d60 0x7fc9a41a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a8e00640 1 -- 172.21.15.46:0/896236962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a41a20d0 con 0x7fc9a41041b0 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a9800640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a4105d60 0x7fc9a41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a8e00640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a41041b0 0x7fc9a419a4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc998009230 tx=0x7fc998005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a1600640 1 -- 172.21.15.46:0/896236962 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998041070 con 0x7fc9a41041b0 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a41a2360 con 0x7fc9a41041b0 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.290+0000 7fc9a1600640 1 -- 172.21.15.46:0/896236962 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9980054e0 con 0x7fc9a41041b0 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.291+0000 7fc9a1600640 1 -- 172.21.15.46:0/896236962 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998034440 con 0x7fc9a41041b0 2024-04-17T07:55:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.291+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a41a28d0 con 0x7fc9a41041b0 2024-04-17T07:55:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.292+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a410b4d0 con 0x7fc9a41041b0 2024-04-17T07:55:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.294+0000 7fc9a1600640 1 -- 172.21.15.46:0/896236962 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc998044070 con 0x7fc9a41041b0 2024-04-17T07:55:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.295+0000 7fc9a1600640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc98c0a3bb0 0x7fc98c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.296+0000 7fc9a1600640 1 -- 172.21.15.46:0/896236962 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc9980f2370 con 0x7fc9a41041b0 2024-04-17T07:55:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.296+0000 7fc9a3e00640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc98c0a3bb0 0x7fc98c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.296+0000 7fc9a3e00640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc98c0a3bb0 0x7fc98c0a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9a419b9c0 tx=0x7fc99000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:55.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.298+0000 7fc9a1600640 1 -- 172.21.15.46:0/896236962 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc9980be460 con 0x7fc9a41041b0 2024-04-17T07:55:55.359 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:55:55 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:55:55.037+0000 7fc222a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:55:55.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:55:55 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:55:55.037+0000 7fc222a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:55:55.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:55:55 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:55:55.037+0000 7fc222a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:55:55.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:55:55 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:55:55.037+0000 7fc222a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:55:55.360 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 07:55:55 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T07:55:55.037+0000 7fc222a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:55:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.534+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc9a40623a0 con 0x7fc98c0a3bb0 2024-04-17T07:55:55.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.578+0000 7fc9a1600640 1 -- 172.21.15.46:0/896236962 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9a40623a0 con 0x7fc98c0a3bb0 2024-04-17T07:55:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.580+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc98c0a3bb0 msgr2=0x7fc98c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.580+0000 7fc9ab800640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc98c0a3bb0 0x7fc98c0a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9a419b9c0 tx=0x7fc99000b410 comp rx=0 tx=0).stop 2024-04-17T07:55:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.580+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a41041b0 msgr2=0x7fc9a419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.580+0000 7fc9ab800640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a41041b0 0x7fc9a419a4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc998009230 tx=0x7fc998005dd0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 shutdown_connections 2024-04-17T07:55:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc98c0a3bb0 0x7fc98c0a6070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a4105d60 0x7fc9a41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9a41053a0 0x7fc9a419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 --2- 172.21.15.46:0/896236962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9a41041b0 0x7fc9a419a4b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 >> 172.21.15.46:0/896236962 conn(0x7fc9a40ff720 msgr2=0x7fc9a41086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 shutdown_connections 2024-04-17T07:55:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.581+0000 7fc9ab800640 1 -- 172.21.15.46:0/896236962 wait complete. 2024-04-17T07:55:55.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-17T07:55:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 -- 172.21.15.46:0/3455214836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b181011d0 msgr2=0x7f9b18101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 --2- 172.21.15.46:0/3455214836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b181011d0 0x7f9b18101630 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009900 tx=0x7f9b0002bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 -- 172.21.15.46:0/3455214836 shutdown_connections 2024-04-17T07:55:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 --2- 172.21.15.46:0/3455214836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18101b70 0x7f9b18101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 --2- 172.21.15.46:0/3455214836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b181011d0 0x7f9b18101630 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 --2- 172.21.15.46:0/3455214836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b18100870 0x7f9b18100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 -- 172.21.15.46:0/3455214836 >> 172.21.15.46:0/3455214836 conn(0x7f9b180fc430 msgr2=0x7f9b180fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.836+0000 7f9b1e000640 1 -- 172.21.15.46:0/3455214836 shutdown_connections 2024-04-17T07:55:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.837+0000 7f9b1e000640 1 -- 172.21.15.46:0/3455214836 wait complete. 2024-04-17T07:55:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 Processor -- start 2024-04-17T07:55:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 -- start start 2024-04-17T07:55:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18100870 0x7f9b1819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b181011d0 0x7f9b1819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b18101b70 0x7f9b181a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1810fe90 con 0x7f9b181011d0 2024-04-17T07:55:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1810fd10 con 0x7f9b18101b70 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.838+0000 7f9b1e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9b18110010 con 0x7f9b18100870 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b181011d0 0x7f9b1819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18100870 0x7f9b1819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18100870 0x7f9b1819e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36876/0 (socket says 172.21.15.46:36876) 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 -- 172.21.15.46:0/521751824 learned_addr learned my addr 172.21.15.46:0/521751824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17e00640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b18101b70 0x7f9b181a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 -- 172.21.15.46:0/521751824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b18101b70 msgr2=0x7f9b181a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b18101b70 0x7f9b181a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 -- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b181011d0 msgr2=0x7f9b1819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b181011d0 0x7f9b1819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 -- 172.21.15.46:0/521751824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b181a6500 con 0x7f9b18100870 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b16a00640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b181011d0 0x7f9b1819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.839+0000 7f9b17400640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18100870 0x7f9b1819e8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009d50 tx=0x7f9b0800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.840+0000 7f9b0fe00640 1 -- 172.21.15.46:0/521751824 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08003b70 con 0x7f9b18100870 2024-04-17T07:55:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.840+0000 7f9b0fe00640 1 -- 172.21.15.46:0/521751824 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b08003d10 con 0x7f9b18100870 2024-04-17T07:55:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.840+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b181a67f0 con 0x7f9b18100870 2024-04-17T07:55:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.840+0000 7f9b0fe00640 1 -- 172.21.15.46:0/521751824 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080108f0 con 0x7f9b18100870 2024-04-17T07:55:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.840+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b181a6d60 con 0x7f9b18100870 2024-04-17T07:55:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.841+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b181071b0 con 0x7f9b18100870 2024-04-17T07:55:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.843+0000 7f9b0fe00640 1 -- 172.21.15.46:0/521751824 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b08004000 con 0x7f9b18100870 2024-04-17T07:55:55.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.844+0000 7f9b0fe00640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9afc0a3cc0 0x7f9afc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:55.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.844+0000 7f9b0fe00640 1 -- 172.21.15.46:0/521751824 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9b080d0dd0 con 0x7f9b18100870 2024-04-17T07:55:55.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.845+0000 7f9b16a00640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9afc0a3cc0 0x7f9afc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:55.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.846+0000 7f9b16a00640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9afc0a3cc0 0x7f9afc0a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002c250 tx=0x7f9b0002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:55.849+0000 7f9b0fe00640 1 -- 172.21.15.46:0/521751824 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b0809d3b0 con 0x7f9b18100870 2024-04-17T07:55:55.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:55 smithi046 ceph-mon[32038]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:56.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:55 smithi062 ceph-mon[36507]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:56.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.078+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9b180623a0 con 0x7f9afc0a3cc0 2024-04-17T07:55:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.090+0000 7f9b0fe00640 1 -- 172.21.15.46:0/521751824 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9b180623a0 con 0x7f9afc0a3cc0 2024-04-17T07:55:56.091 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_0/d8dbfdec-b6d5-4571-b84d-8ee050e4a489 2024-04-17T07:55:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9afc0a3cc0 msgr2=0x7f9afc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9afc0a3cc0 0x7f9afc0a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002c250 tx=0x7f9b0002cf90 comp rx=0 tx=0).stop 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18100870 msgr2=0x7f9b1819e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18100870 0x7f9b1819e8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009d50 tx=0x7f9b0800c6a0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 shutdown_connections 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9afc0a3cc0 0x7f9afc0a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b18101b70 0x7f9b181a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b181011d0 0x7f9b1819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 --2- 172.21.15.46:0/521751824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b18100870 0x7f9b1819e8e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 >> 172.21.15.46:0/521751824 conn(0x7f9b180fc430 msgr2=0x7f9b180ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:56.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 shutdown_connections 2024-04-17T07:55:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.093+0000 7f9b1e000640 1 -- 172.21.15.46:0/521751824 wait complete. 2024-04-17T07:55:56.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-17T07:55:56.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:55 smithi053 ceph-mon[36437]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 -- 172.21.15.46:0/4139048692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 msgr2=0x7f1994103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 --2- 172.21.15.46:0/4139048692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f1994103460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f197c009900 tx=0x7f197c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 -- 172.21.15.46:0/4139048692 shutdown_connections 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 --2- 172.21.15.46:0/4139048692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19941039a0 0x7f199410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 --2- 172.21.15.46:0/4139048692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f1994103460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 --2- 172.21.15.46:0/4139048692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101df0 0x7f1994102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 -- 172.21.15.46:0/4139048692 >> 172.21.15.46:0/4139048692 conn(0x7f19940fd360 msgr2=0x7f19940ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 -- 172.21.15.46:0/4139048692 shutdown_connections 2024-04-17T07:55:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.337+0000 7f1993e00640 1 -- 172.21.15.46:0/4139048692 wait complete. 2024-04-17T07:55:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.338+0000 7f1993e00640 1 Processor -- start 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.338+0000 7f1993e00640 1 -- start start 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1993e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101df0 0x7f199419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1993e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f199419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1993e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19941039a0 0x7f19941a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1993e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f199410daa0 con 0x7f1994102fe0 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1993e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f199410d920 con 0x7f1994101df0 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1993e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f199410dc20 con 0x7f19941039a0 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f199419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f199419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52276/0 (socket says 172.21.15.46:52276) 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101df0 0x7f199419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 -- 172.21.15.46:0/3493991717 learned_addr learned my addr 172.21.15.46:0/3493991717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1993400640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19941039a0 0x7f19941a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 -- 172.21.15.46:0/3493991717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19941039a0 msgr2=0x7f19941a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19941039a0 0x7f19941a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 -- 172.21.15.46:0/3493991717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101df0 msgr2=0x7f199419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101df0 0x7f199419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.339+0000 7f1992000640 1 -- 172.21.15.46:0/3493991717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19941a1f50 con 0x7f1994102fe0 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.340+0000 7f1992000640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f199419a8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f197c02c260 tx=0x7f197c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.340+0000 7f198b400640 1 -- 172.21.15.46:0/3493991717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c041070 con 0x7f1994102fe0 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.340+0000 7f198b400640 1 -- 172.21.15.46:0/3493991717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f197c0054e0 con 0x7f1994102fe0 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.340+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19941a21e0 con 0x7f1994102fe0 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.340+0000 7f198b400640 1 -- 172.21.15.46:0/3493991717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c0346a0 con 0x7f1994102fe0 2024-04-17T07:55:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.340+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19941a2780 con 0x7f1994102fe0 2024-04-17T07:55:56.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.341+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19941a29e0 con 0x7f1994102fe0 2024-04-17T07:55:56.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.343+0000 7f198b400640 1 -- 172.21.15.46:0/3493991717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f197c044070 con 0x7f1994102fe0 2024-04-17T07:55:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.344+0000 7f198b400640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f19780a3cc0 0x7f19780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.345+0000 7f198b400640 1 -- 172.21.15.46:0/3493991717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f197c0f2210 con 0x7f1994102fe0 2024-04-17T07:55:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.345+0000 7f1992a00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f19780a3cc0 0x7f19780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.345+0000 7f1992a00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f19780a3cc0 0x7f19780a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1984005fd0 tx=0x7f198400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:56.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.349+0000 7f198b400640 1 -- 172.21.15.46:0/3493991717 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f197c003e20 con 0x7f1994102fe0 2024-04-17T07:55:56.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.586+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f19940758b0 con 0x7f19780a3cc0 2024-04-17T07:55:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.627+0000 7f198b400640 1 -- 172.21.15.46:0/3493991717 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19940758b0 con 0x7f19780a3cc0 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f19780a3cc0 msgr2=0x7f19780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f19780a3cc0 0x7f19780a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1984005fd0 tx=0x7f198400b410 comp rx=0 tx=0).stop 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 msgr2=0x7f199419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f199419a8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f197c02c260 tx=0x7f197c005dd0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 shutdown_connections 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f19780a3cc0 0x7f19780a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19941039a0 0x7f19941a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994102fe0 0x7f199419a8c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 --2- 172.21.15.46:0/3493991717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1994101df0 0x7f199419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 >> 172.21.15.46:0/3493991717 conn(0x7f19940fd360 msgr2=0x7f1994074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 shutdown_connections 2024-04-17T07:55:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.630+0000 7f1993e00640 1 -- 172.21.15.46:0/3493991717 wait complete. 2024-04-17T07:55:56.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-17T07:55:56.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 -- 172.21.15.46:0/445887876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 msgr2=0x7f42240743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 --2- 172.21.15.46:0/445887876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f42240743f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4214009900 tx=0x7f421402bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 -- 172.21.15.46:0/445887876 shutdown_connections 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 --2- 172.21.15.46:0/445887876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f422410e3b0 0x7f42241107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 --2- 172.21.15.46:0/445887876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42240749c0 0x7f4224074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 --2- 172.21.15.46:0/445887876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f42240743f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 -- 172.21.15.46:0/445887876 >> 172.21.15.46:0/445887876 conn(0x7f42240fd370 msgr2=0x7f42240ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 -- 172.21.15.46:0/445887876 shutdown_connections 2024-04-17T07:55:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.880+0000 7f4228e00640 1 -- 172.21.15.46:0/445887876 wait complete. 2024-04-17T07:55:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.881+0000 7f4228e00640 1 Processor -- start 2024-04-17T07:55:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.881+0000 7f4228e00640 1 -- start start 2024-04-17T07:55:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.881+0000 7f4228e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42240749c0 0x7f422419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.881+0000 7f4228e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f422419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4228e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f422410e3b0 0x7f42241a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4228e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4224114080 con 0x7f422410e3b0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4228e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4224113f00 con 0x7f42240749c0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4228e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4224114200 con 0x7f4224075fa0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f422419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f422419ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36918/0 (socket says 172.21.15.46:36918) 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 -- 172.21.15.46:0/3643481706 learned_addr learned my addr 172.21.15.46:0/3643481706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f422410e3b0 0x7f42241a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4223400640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42240749c0 0x7f422419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 -- 172.21.15.46:0/3643481706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42240749c0 msgr2=0x7f422419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42240749c0 0x7f422419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 -- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f422410e3b0 msgr2=0x7f42241a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f422410e3b0 0x7f42241a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.882+0000 7f4222a00640 1 -- 172.21.15.46:0/3643481706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42241a62f0 con 0x7f4224075fa0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.883+0000 7f4222a00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f422419ec10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f421000e9f0 tx=0x7f421000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.883+0000 7f421be00640 1 -- 172.21.15.46:0/3643481706 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421000cd80 con 0x7f4224075fa0 2024-04-17T07:55:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.883+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42241a65e0 con 0x7f4224075fa0 2024-04-17T07:55:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.883+0000 7f421be00640 1 -- 172.21.15.46:0/3643481706 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4210010910 con 0x7f4224075fa0 2024-04-17T07:55:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.883+0000 7f421be00640 1 -- 172.21.15.46:0/3643481706 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42100188f0 con 0x7f4224075fa0 2024-04-17T07:55:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.883+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42241a6b30 con 0x7f4224075fa0 2024-04-17T07:55:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.884+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4224104580 con 0x7f4224075fa0 2024-04-17T07:55:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.887+0000 7f421be00640 1 -- 172.21.15.46:0/3643481706 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4210010ac0 con 0x7f4224075fa0 2024-04-17T07:55:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.888+0000 7f421be00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f42080a3c40 0x7f42080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.888+0000 7f421be00640 1 -- 172.21.15.46:0/3643481706 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f42100d0bc0 con 0x7f4224075fa0 2024-04-17T07:55:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.889+0000 7f4223400640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f42080a3c40 0x7f42080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.889+0000 7f4223400640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f42080a3c40 0x7f42080a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f421402c250 tx=0x7f4214000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:56.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:56.893+0000 7f421be00640 1 -- 172.21.15.46:0/3643481706 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f421009cb50 con 0x7f4224075fa0 2024-04-17T07:55:56.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:56 smithi053 ceph-mon[36437]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-17T07:55:56.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:56 smithi053 ceph-mon[36437]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:56.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:56 smithi053 ceph-mon[36437]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-17T07:55:56.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:56 smithi053 ceph-mon[36437]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:56.972 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:56 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: 2024-04-17T07:55:56.587+0000 7f2180e94700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-17T07:55:56.972 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 07:55:56 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: 2024-04-17T07:55:56.587+0000 7f2180e94700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-17T07:55:57.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:56 smithi062 ceph-mon[36507]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-17T07:55:57.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:56 smithi062 ceph-mon[36507]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:57.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:56 smithi062 ceph-mon[36507]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-17T07:55:57.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:56 smithi062 ceph-mon[36507]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:57.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:56 smithi046 ceph-mon[32038]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-17T07:55:57.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:56 smithi046 ceph-mon[32038]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:57.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:56 smithi046 ceph-mon[32038]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-17T07:55:57.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:56 smithi046 ceph-mon[32038]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:57.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.132+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4224062430 con 0x7f42080a3c40 2024-04-17T07:55:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.141+0000 7f421be00640 1 -- 172.21.15.46:0/3643481706 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4224062430 con 0x7f42080a3c40 2024-04-17T07:55:57.142 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_1/1cdce49c-fba7-4c4b-8cb5-c9dc1266c1ad 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f42080a3c40 msgr2=0x7f42080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f42080a3c40 0x7f42080a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f421402c250 tx=0x7f4214000bc0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 msgr2=0x7f422419ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f422419ec10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f421000e9f0 tx=0x7f421000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 shutdown_connections 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f42080a3c40 0x7f42080a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f422410e3b0 0x7f42241a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4224075fa0 0x7f422419ec10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 --2- 172.21.15.46:0/3643481706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42240749c0 0x7f422419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.144+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 >> 172.21.15.46:0/3643481706 conn(0x7f42240fd370 msgr2=0x7f4224101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.145+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 shutdown_connections 2024-04-17T07:55:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.145+0000 7f4228e00640 1 -- 172.21.15.46:0/3643481706 wait complete. 2024-04-17T07:55:57.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:55:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.393+0000 7f6267200640 1 -- 172.21.15.46:0/3303372197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 msgr2=0x7f6268102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.393+0000 7f6267200640 1 --2- 172.21.15.46:0/3303372197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 0x7f6268102400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6254009860 tx=0x7f625402bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.393+0000 7f6267200640 1 -- 172.21.15.46:0/3303372197 shutdown_connections 2024-04-17T07:55:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.394+0000 7f6267200640 1 --2- 172.21.15.46:0/3303372197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6268102940 0x7f62681091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.394+0000 7f6267200640 1 --2- 172.21.15.46:0/3303372197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 0x7f6268102400 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.394+0000 7f6267200640 1 --2- 172.21.15.46:0/3303372197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 0x7f62681011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.394+0000 7f6267200640 1 -- 172.21.15.46:0/3303372197 >> 172.21.15.46:0/3303372197 conn(0x7f62680fc300 msgr2=0x7f62680fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.394+0000 7f6267200640 1 -- 172.21.15.46:0/3303372197 shutdown_connections 2024-04-17T07:55:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.394+0000 7f6267200640 1 -- 172.21.15.46:0/3303372197 wait complete. 2024-04-17T07:55:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.395+0000 7f6267200640 1 Processor -- start 2024-04-17T07:55:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.395+0000 7f6267200640 1 -- start start 2024-04-17T07:55:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6267200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 0x7f62681981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6267200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 0x7f6268198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6267200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6268102940 0x7f626819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6267200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f626810db30 con 0x7f6268100d90 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6267200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f626810d9b0 con 0x7f6268101f80 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6267200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f626810dcb0 con 0x7f6268102940 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 0x7f62681981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 0x7f62681981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52322/0 (socket says 172.21.15.46:52322) 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265e00640 1 -- 172.21.15.46:0/476480424 learned_addr learned my addr 172.21.15.46:0/476480424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265400640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 0x7f6268198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6266800640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6268102940 0x7f626819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265400640 1 -- 172.21.15.46:0/476480424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6268102940 msgr2=0x7f626819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265400640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6268102940 0x7f626819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265400640 1 -- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 msgr2=0x7f62681981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265400640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 0x7f62681981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265400640 1 -- 172.21.15.46:0/476480424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626819fe00 con 0x7f6268101f80 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6265e00640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 0x7f62681981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.396+0000 7f6266800640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6268102940 0x7f626819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.397+0000 7f6265400640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 0x7f6268198720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f625402c0b0 tx=0x7f62540042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.397+0000 7f625ea00640 1 -- 172.21.15.46:0/476480424 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254041020 con 0x7f6268101f80 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.397+0000 7f625ea00640 1 -- 172.21.15.46:0/476480424 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62540323b0 con 0x7f6268101f80 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.397+0000 7f625ea00640 1 -- 172.21.15.46:0/476480424 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254039040 con 0x7f6268101f80 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.397+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62681a0090 con 0x7f6268101f80 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.397+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62681a0500 con 0x7f6268101f80 2024-04-17T07:55:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.398+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62681080b0 con 0x7f6268101f80 2024-04-17T07:55:57.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.400+0000 7f625ea00640 1 -- 172.21.15.46:0/476480424 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6254044020 con 0x7f6268101f80 2024-04-17T07:55:57.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.402+0000 7f625ea00640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f624c0a3cb0 0x7f624c0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.403+0000 7f625ea00640 1 -- 172.21.15.46:0/476480424 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f62540f1f90 con 0x7f6268101f80 2024-04-17T07:55:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.403+0000 7f6265e00640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f624c0a3cb0 0x7f624c0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.403+0000 7f6265e00640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f624c0a3cb0 0x7f624c0a6170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6258003b00 tx=0x7f6258010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.406+0000 7f625ea00640 1 -- 172.21.15.46:0/476480424 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f625404b020 con 0x7f6268101f80 2024-04-17T07:55:57.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.660+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f626802d820 con 0x7f6268101f80 2024-04-17T07:55:57.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.662+0000 7f625ea00640 1 -- 172.21.15.46:0/476480424 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f62540be680 con 0x7f6268101f80 2024-04-17T07:55:57.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:57.663 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:54.519826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24556,24542]},"id":1}]} 2024-04-17T07:55:57.663 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-04-17T07:55:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f624c0a3cb0 msgr2=0x7f624c0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f624c0a3cb0 0x7f624c0a6170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6258003b00 tx=0x7f6258010040 comp rx=0 tx=0).stop 2024-04-17T07:55:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 msgr2=0x7f6268198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 0x7f6268198720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f625402c0b0 tx=0x7f62540042f0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 shutdown_connections 2024-04-17T07:55:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f624c0a3cb0 0x7f624c0a6170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6268102940 0x7f626819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6268101f80 0x7f6268198720 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 --2- 172.21.15.46:0/476480424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6268100d90 0x7f62681981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 >> 172.21.15.46:0/476480424 conn(0x7f62680fc300 msgr2=0x7f6268105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 shutdown_connections 2024-04-17T07:55:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.665+0000 7f6267200640 1 -- 172.21.15.46:0/476480424 wait complete. 2024-04-17T07:55:57.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 -- 172.21.15.46:0/2042472327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8104240 msgr2=0x7f63b8104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 --2- 172.21.15.46:0/2042472327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8104240 0x7f63b8104660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009900 tx=0x7f63a802bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 -- 172.21.15.46:0/2042472327 shutdown_connections 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 --2- 172.21.15.46:0/2042472327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8105df0 0x7f63b810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 --2- 172.21.15.46:0/2042472327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b8105430 0x7f63b81058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 --2- 172.21.15.46:0/2042472327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8104240 0x7f63b8104660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 -- 172.21.15.46:0/2042472327 >> 172.21.15.46:0/2042472327 conn(0x7f63b80ff7b0 msgr2=0x7f63b8101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.903+0000 7f63bf000640 1 -- 172.21.15.46:0/2042472327 shutdown_connections 2024-04-17T07:55:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.904+0000 7f63bf000640 1 -- 172.21.15.46:0/2042472327 wait complete. 2024-04-17T07:55:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.905+0000 7f63bf000640 1 Processor -- start 2024-04-17T07:55:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.905+0000 7f63bf000640 1 -- start start 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.905+0000 7f63bf000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b8104240 0x7f63b819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.905+0000 7f63bf000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8105430 0x7f63b819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.905+0000 7f63bf000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8105df0 0x7f63b81a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63bf000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63b810fe80 con 0x7f63b8105df0 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63bf000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f63b810fd00 con 0x7f63b8104240 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63bf000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8110000 con 0x7f63b8105430 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8105430 0x7f63b819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8105430 0x7f63b819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:54088/0 (socket says 172.21.15.46:54088) 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63bd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8105df0 0x7f63b81a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 -- 172.21.15.46:0/3018536782 learned_addr learned my addr 172.21.15.46:0/3018536782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63b7e00640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b8104240 0x7f63b819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 -- 172.21.15.46:0/3018536782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b8104240 msgr2=0x7f63b819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b8104240 0x7f63b819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 -- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8105df0 msgr2=0x7f63b81a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8105df0 0x7f63b81a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63af400640 1 -- 172.21.15.46:0/3018536782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b81a2110 con 0x7f63b8105430 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.906+0000 7f63bd000640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8105df0 0x7f63b81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.907+0000 7f63af400640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8105430 0x7f63b819aa30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63a000e9f0 tx=0x7f63a000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.907+0000 7f63b5600640 1 -- 172.21.15.46:0/3018536782 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a000ccc0 con 0x7f63b8105430 2024-04-17T07:55:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.907+0000 7f63b5600640 1 -- 172.21.15.46:0/3018536782 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0010910 con 0x7f63b8105430 2024-04-17T07:55:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.907+0000 7f63b5600640 1 -- 172.21.15.46:0/3018536782 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00188a0 con 0x7f63b8105430 2024-04-17T07:55:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.907+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b81a2400 con 0x7f63b8105430 2024-04-17T07:55:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.908+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b81a2900 con 0x7f63b8105430 2024-04-17T07:55:57.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.909+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b810b560 con 0x7f63b8105430 2024-04-17T07:55:57.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.910+0000 7f63b5600640 1 -- 172.21.15.46:0/3018536782 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f63a0010ac0 con 0x7f63b8105430 2024-04-17T07:55:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.912+0000 7f63b5600640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f639c0a3cc0 0x7f639c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.912+0000 7f63b5600640 1 -- 172.21.15.46:0/3018536782 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f63a00d0b60 con 0x7f63b8105430 2024-04-17T07:55:57.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.912+0000 7f63b7e00640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f639c0a3cc0 0x7f639c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.913+0000 7f63b7e00640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f639c0a3cc0 0x7f639c0a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63a802c180 tx=0x7f63a802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:57.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:57.916+0000 7f63b5600640 1 -- 172.21.15.46:0/3018536782 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f63a009d1a0 con 0x7f63b8105430 2024-04-17T07:55:58.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:57 smithi062 ceph-mon[36507]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:58.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:57 smithi062 ceph-mon[36507]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:58.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:57 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/476480424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:58.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:57 smithi046 ceph-mon[32038]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:58.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:57 smithi046 ceph-mon[32038]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:58.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:57 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/476480424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.126+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63b8062b60 con 0x7f63b8105430 2024-04-17T07:55:58.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.129+0000 7f63b5600640 1 -- 172.21.15.46:0/3018536782 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f63a009caa0 con 0x7f63b8105430 2024-04-17T07:55:58.129 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:58.129 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f639c0a3cc0 msgr2=0x7f639c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f639c0a3cc0 0x7f639c0a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63a802c180 tx=0x7f63a802cf90 comp rx=0 tx=0).stop 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8105430 msgr2=0x7f63b819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8105430 0x7f63b819aa30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63a000e9f0 tx=0x7f63a000edf0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 shutdown_connections 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f639c0a3cc0 0x7f639c0a6180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8105df0 0x7f63b81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8105430 0x7f63b819aa30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 --2- 172.21.15.46:0/3018536782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63b8104240 0x7f63b819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 >> 172.21.15.46:0/3018536782 conn(0x7f63b80ff7b0 msgr2=0x7f63b8108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.131+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 shutdown_connections 2024-04-17T07:55:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.132+0000 7f63bf000640 1 -- 172.21.15.46:0/3018536782 wait complete. 2024-04-17T07:55:58.153 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T07:55:58.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:55:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:57 smithi053 ceph-mon[36437]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:57 smithi053 ceph-mon[36437]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:55:58.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:57 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/476480424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:55:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 -- 172.21.15.46:0/2318153921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 msgr2=0x7f9484109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 --2- 172.21.15.46:0/2318153921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f9484109c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9470009900 tx=0x7f947002bc40 comp rx=0 tx=0).stop 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 -- 172.21.15.46:0/2318153921 shutdown_connections 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 --2- 172.21.15.46:0/2318153921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f948410a190 0x7f9484110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 --2- 172.21.15.46:0/2318153921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f9484109c50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 --2- 172.21.15.46:0/2318153921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94841085e0 0x7f9484108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 -- 172.21.15.46:0/2318153921 >> 172.21.15.46:0/2318153921 conn(0x7f94840775b0 msgr2=0x7f94840799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 -- 172.21.15.46:0/2318153921 shutdown_connections 2024-04-17T07:55:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.361+0000 7f9489400640 1 -- 172.21.15.46:0/2318153921 wait complete. 2024-04-17T07:55:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.362+0000 7f9489400640 1 Processor -- start 2024-04-17T07:55:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.362+0000 7f9489400640 1 -- start start 2024-04-17T07:55:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9489400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94841085e0 0x7f948419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9489400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f948419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9489400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948410a190 0x7f94841a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9489400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9484114260 con 0x7f94841085e0 2024-04-17T07:55:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9489400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f94841140e0 con 0x7f94841097d0 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9489400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f94841143e0 con 0x7f948410a190 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f948419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f948419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50592/0 (socket says 172.21.15.46:50592) 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 -- 172.21.15.46:0/2379559289 learned_addr learned my addr 172.21.15.46:0/2379559289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9482800640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94841085e0 0x7f948419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9483200640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948410a190 0x7f94841a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 -- 172.21.15.46:0/2379559289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948410a190 msgr2=0x7f94841a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948410a190 0x7f94841a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 -- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94841085e0 msgr2=0x7f948419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94841085e0 0x7f948419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.363+0000 7f9481e00640 1 -- 172.21.15.46:0/2379559289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94841a64f0 con 0x7f94841097d0 2024-04-17T07:55:58.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.364+0000 7f9481e00640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f948419ee10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9470009230 tx=0x7f9470005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:58.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.364+0000 7f947b400640 1 -- 172.21.15.46:0/2379559289 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470041070 con 0x7f94841097d0 2024-04-17T07:55:58.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.364+0000 7f947b400640 1 -- 172.21.15.46:0/2379559289 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94700044d0 con 0x7f94841097d0 2024-04-17T07:55:58.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.364+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94841a6780 con 0x7f94841097d0 2024-04-17T07:55:58.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.365+0000 7f947b400640 1 -- 172.21.15.46:0/2379559289 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470034440 con 0x7f94841097d0 2024-04-17T07:55:58.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.365+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94841a6cc0 con 0x7f94841097d0 2024-04-17T07:55:58.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.366+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948410f900 con 0x7f94841097d0 2024-04-17T07:55:58.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.368+0000 7f947b400640 1 -- 172.21.15.46:0/2379559289 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9470044070 con 0x7f94841097d0 2024-04-17T07:55:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.370+0000 7f947b400640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f94680a3cc0 0x7f94680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.370+0000 7f947b400640 1 -- 172.21.15.46:0/2379559289 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f94700f2240 con 0x7f94841097d0 2024-04-17T07:55:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.370+0000 7f9482800640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f94680a3cc0 0x7f94680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.371+0000 7f9482800640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f94680a3cc0 0x7f94680a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9484109630 tx=0x7f9474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:58.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.374+0000 7f947b400640 1 -- 172.21.15.46:0/2379559289 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f94700be2e0 con 0x7f94841097d0 2024-04-17T07:55:58.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.575+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94840623a0 con 0x7f94680a3cc0 2024-04-17T07:55:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.589+0000 7f947b400640 1 -- 172.21.15.46:0/2379559289 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f94840623a0 con 0x7f94680a3cc0 2024-04-17T07:55:58.595 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f94680a3cc0 msgr2=0x7f94680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f94680a3cc0 0x7f94680a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9484109630 tx=0x7f9474005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 msgr2=0x7f948419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f948419ee10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9470009230 tx=0x7f9470005d50 comp rx=0 tx=0).stop 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 shutdown_connections 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f94680a3cc0 0x7f94680a6180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948410a190 0x7f94841a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94841097d0 0x7f948419ee10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 --2- 172.21.15.46:0/2379559289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94841085e0 0x7f948419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 >> 172.21.15.46:0/2379559289 conn(0x7f94840775b0 msgr2=0x7f948410cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 shutdown_connections 2024-04-17T07:55:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.595+0000 7f9489400640 1 -- 172.21.15.46:0/2379559289 wait complete. 2024-04-17T07:55:58.615 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-17T07:55:57.514390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636666,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":160,"num_read_kb":157,"num_write":316,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":255,"ondisk_log_size":255,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375196,"kb_used_data":13548,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566628,"statfs":{"total":1151940427776,"available":1151556227072,"internally_reserved":0,"allocated":13873152,"data_stored":7592257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43439,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":34,"num_read_kb":48,"num_write":89,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005977"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540145+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:55:32.540145+0000","last_peered":"2024-04-17T07:55:32.540145+0000","last_clean":"2024-04-17T07:55:32.540145+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:55:32.540145+0000","last_undegraded":"2024-04-17T07:55:32.540145+0000","last_fullsized":"2024-04-17T07:55:32.540145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541318+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:55:32.541318+0000","last_peered":"2024-04-17T07:55:32.541318+0000","last_clean":"2024-04-17T07:55:32.541318+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:55:32.541318+0000","last_undegraded":"2024-04-17T07:55:32.541318+0000","last_fullsized":"2024-04-17T07:55:32.541318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540623+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:55:32.540623+0000","last_peered":"2024-04-17T07:55:32.540623+0000","last_clean":"2024-04-17T07:55:32.540623+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:55:32.540623+0000","last_undegraded":"2024-04-17T07:55:32.540623+0000","last_fullsized":"2024-04-17T07:55:32.540623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541526+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:55:32.541526+0000","last_peered":"2024-04-17T07:55:32.541526+0000","last_clean":"2024-04-17T07:55:32.541526+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:55:32.541526+0000","last_undegraded":"2024-04-17T07:55:32.541526+0000","last_fullsized":"2024-04-17T07:55:32.541526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540948+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:55:32.540948+0000","last_peered":"2024-04-17T07:55:32.540948+0000","last_clean":"2024-04-17T07:55:32.540948+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:55:32.540948+0000","last_undegraded":"2024-04-17T07:55:32.540948+0000","last_fullsized":"2024-04-17T07:55:32.540948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540735+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:55:32.540735+0000","last_peered":"2024-04-17T07:55:32.540735+0000","last_clean":"2024-04-17T07:55:32.540735+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:55:32.540735+0000","last_undegraded":"2024-04-17T07:55:32.540735+0000","last_fullsized":"2024-04-17T07:55:32.540735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604045+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:55:31.604045+0000","last_peered":"2024-04-17T07:55:31.604045+0000","last_clean":"2024-04-17T07:55:31.604045+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:55:31.604045+0000","last_undegraded":"2024-04-17T07:55:31.604045+0000","last_fullsized":"2024-04-17T07:55:31.604045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779511+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:55:31.779511+0000","last_peered":"2024-04-17T07:55:31.779511+0000","last_clean":"2024-04-17T07:55:31.779511+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:55:31.779511+0000","last_undegraded":"2024-04-17T07:55:31.779511+0000","last_fullsized":"2024-04-17T07:55:31.779511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540539+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:55:32.540539+0000","last_peered":"2024-04-17T07:55:32.540539+0000","last_clean":"2024-04-17T07:55:32.540539+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:55:32.540539+0000","last_undegraded":"2024-04-17T07:55:32.540539+0000","last_fullsized":"2024-04-17T07:55:32.540539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541050+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:55:32.541050+0000","last_peered":"2024-04-17T07:55:32.541050+0000","last_clean":"2024-04-17T07:55:32.541050+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:55:32.541050+0000","last_undegraded":"2024-04-17T07:55:32.541050+0000","last_fullsized":"2024-04-17T07:55:32.541050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604781+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:55:31.604781+0000","last_peered":"2024-04-17T07:55:31.604781+0000","last_clean":"2024-04-17T07:55:31.604781+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:55:31.604781+0000","last_undegraded":"2024-04-17T07:55:31.604781+0000","last_fullsized":"2024-04-17T07:55:31.604781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541330+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:55:32.541330+0000","last_peered":"2024-04-17T07:55:32.541330+0000","last_clean":"2024-04-17T07:55:32.541330+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:55:32.541330+0000","last_undegraded":"2024-04-17T07:55:32.541330+0000","last_fullsized":"2024-04-17T07:55:32.541330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606373+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:55:31.606373+0000","last_peered":"2024-04-17T07:55:31.606373+0000","last_clean":"2024-04-17T07:55:31.606373+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:55:31.606373+0000","last_undegraded":"2024-04-17T07:55:31.606373+0000","last_fullsized":"2024-04-17T07:55:31.606373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539861+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:55:32.539861+0000","last_peered":"2024-04-17T07:55:32.539861+0000","last_clean":"2024-04-17T07:55:32.539861+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:55:32.539861+0000","last_undegraded":"2024-04-17T07:55:32.539861+0000","last_fullsized":"2024-04-17T07:55:32.539861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540270+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:55:32.540270+0000","last_peered":"2024-04-17T07:55:32.540270+0000","last_clean":"2024-04-17T07:55:32.540270+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:55:32.540270+0000","last_undegraded":"2024-04-17T07:55:32.540270+0000","last_fullsized":"2024-04-17T07:55:32.540270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539579+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:55:32.539579+0000","last_peered":"2024-04-17T07:55:32.539579+0000","last_clean":"2024-04-17T07:55:32.539579+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:55:32.539579+0000","last_undegraded":"2024-04-17T07:55:32.539579+0000","last_fullsized":"2024-04-17T07:55:32.539579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540015+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:55:32.540015+0000","last_peered":"2024-04-17T07:55:32.540015+0000","last_clean":"2024-04-17T07:55:32.540015+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:55:32.540015+0000","last_undegraded":"2024-04-17T07:55:32.540015+0000","last_fullsized":"2024-04-17T07:55:32.540015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.544098+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:55:32.544098+0000","last_peered":"2024-04-17T07:55:32.544098+0000","last_clean":"2024-04-17T07:55:32.544098+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:55:32.544098+0000","last_undegraded":"2024-04-17T07:55:32.544098+0000","last_fullsized":"2024-04-17T07:55:32.544098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609114+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:55:31.609114+0000","last_peered":"2024-04-17T07:55:31.609114+0000","last_clean":"2024-04-17T07:55:31.609114+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:55:31.609114+0000","last_undegraded":"2024-04-17T07:55:31.609114+0000","last_fullsized":"2024-04-17T07:55:31.609114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608992+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:55:31.608992+0000","last_peered":"2024-04-17T07:55:31.608992+0000","last_clean":"2024-04-17T07:55:31.608992+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:55:31.608992+0000","last_undegraded":"2024-04-17T07:55:31.608992+0000","last_fullsized":"2024-04-17T07:55:31.608992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.740250+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:55:53.740250+0000","last_peered":"2024-04-17T07:55:53.740250+0000","last_clean":"2024-04-17T07:55:53.740250+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:55:53.740250+0000","last_undegraded":"2024-04-17T07:55:53.740250+0000","last_fullsized":"2024-04-17T07:55:53.740250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780610+0000","last_change":"2024-04-17T07:55:26.373262+0000","last_active":"2024-04-17T07:55:31.780610+0000","last_peered":"2024-04-17T07:55:31.780610+0000","last_clean":"2024-04-17T07:55:31.780610+0000","last_became_active":"2024-04-17T07:55:26.366248+0000","last_became_peered":"2024-04-17T07:55:26.366248+0000","last_unstale":"2024-04-17T07:55:31.780610+0000","last_undegraded":"2024-04-17T07:55:31.780610+0000","last_fullsized":"2024-04-17T07:55:31.780610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:16.251444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540058+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:55:32.540058+0000","last_peered":"2024-04-17T07:55:32.540058+0000","last_clean":"2024-04-17T07:55:32.540058+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:55:32.540058+0000","last_undegraded":"2024-04-17T07:55:32.540058+0000","last_fullsized":"2024-04-17T07:55:32.540058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.781038+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:55:31.781038+0000","last_peered":"2024-04-17T07:55:31.781038+0000","last_clean":"2024-04-17T07:55:31.781038+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:55:31.781038+0000","last_undegraded":"2024-04-17T07:55:31.781038+0000","last_fullsized":"2024-04-17T07:55:31.781038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540314+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:55:32.540314+0000","last_peered":"2024-04-17T07:55:32.540314+0000","last_clean":"2024-04-17T07:55:32.540314+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:55:32.540314+0000","last_undegraded":"2024-04-17T07:55:32.540314+0000","last_fullsized":"2024-04-17T07:55:32.540314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541095+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:55:32.541095+0000","last_peered":"2024-04-17T07:55:32.541095+0000","last_clean":"2024-04-17T07:55:32.541095+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:55:32.541095+0000","last_undegraded":"2024-04-17T07:55:32.541095+0000","last_fullsized":"2024-04-17T07:55:32.541095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.728146+0000","last_change":"2024-04-17T07:55:25.509157+0000","last_active":"2024-04-17T07:55:53.728146+0000","last_peered":"2024-04-17T07:55:53.728146+0000","last_clean":"2024-04-17T07:55:53.728146+0000","last_became_active":"2024-04-17T07:55:25.508793+0000","last_became_peered":"2024-04-17T07:55:25.508793+0000","last_unstale":"2024-04-17T07:55:53.728146+0000","last_undegraded":"2024-04-17T07:55:53.728146+0000","last_fullsized":"2024-04-17T07:55:53.728146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:55:29.596338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540541+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:55:32.540541+0000","last_peered":"2024-04-17T07:55:32.540541+0000","last_clean":"2024-04-17T07:55:32.540541+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:55:32.540541+0000","last_undegraded":"2024-04-17T07:55:32.540541+0000","last_fullsized":"2024-04-17T07:55:32.540541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539773+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:55:32.539773+0000","last_peered":"2024-04-17T07:55:32.539773+0000","last_clean":"2024-04-17T07:55:32.539773+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:55:32.539773+0000","last_undegraded":"2024-04-17T07:55:32.539773+0000","last_fullsized":"2024-04-17T07:55:32.539773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541148+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:55:32.541148+0000","last_peered":"2024-04-17T07:55:32.541148+0000","last_clean":"2024-04-17T07:55:32.541148+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:55:32.541148+0000","last_undegraded":"2024-04-17T07:55:32.541148+0000","last_fullsized":"2024-04-17T07:55:32.541148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540099+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:55:32.540099+0000","last_peered":"2024-04-17T07:55:32.540099+0000","last_clean":"2024-04-17T07:55:32.540099+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:55:32.540099+0000","last_undegraded":"2024-04-17T07:55:32.540099+0000","last_fullsized":"2024-04-17T07:55:32.540099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609019+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:55:31.609019+0000","last_peered":"2024-04-17T07:55:31.609019+0000","last_clean":"2024-04-17T07:55:31.609019+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:55:31.609019+0000","last_undegraded":"2024-04-17T07:55:31.609019+0000","last_fullsized":"2024-04-17T07:55:31.609019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:55.535908+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:55:55.535908+0000","last_peered":"2024-04-17T07:55:55.535908+0000","last_clean":"2024-04-17T07:55:55.535908+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:55:55.535908+0000","last_undegraded":"2024-04-17T07:55:55.535908+0000","last_fullsized":"2024-04-17T07:55:55.535908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540874+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:55:32.540874+0000","last_peered":"2024-04-17T07:55:32.540874+0000","last_clean":"2024-04-17T07:55:32.540874+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:55:32.540874+0000","last_undegraded":"2024-04-17T07:55:32.540874+0000","last_fullsized":"2024-04-17T07:55:32.540874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609373+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:55:31.609373+0000","last_peered":"2024-04-17T07:55:31.609373+0000","last_clean":"2024-04-17T07:55:31.609373+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:55:31.609373+0000","last_undegraded":"2024-04-17T07:55:31.609373+0000","last_fullsized":"2024-04-17T07:55:31.609373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.730259+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:55:53.730259+0000","last_peered":"2024-04-17T07:55:53.730259+0000","last_clean":"2024-04-17T07:55:53.730259+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:55:53.730259+0000","last_undegraded":"2024-04-17T07:55:53.730259+0000","last_fullsized":"2024-04-17T07:55:53.730259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539842+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:55:32.539842+0000","last_peered":"2024-04-17T07:55:32.539842+0000","last_clean":"2024-04-17T07:55:32.539842+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:55:32.539842+0000","last_undegraded":"2024-04-17T07:55:32.539842+0000","last_fullsized":"2024-04-17T07:55:32.539842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540068+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:55:32.540068+0000","last_peered":"2024-04-17T07:55:32.540068+0000","last_clean":"2024-04-17T07:55:32.540068+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:55:32.540068+0000","last_undegraded":"2024-04-17T07:55:32.540068+0000","last_fullsized":"2024-04-17T07:55:32.540068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.735152+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:55:53.735152+0000","last_peered":"2024-04-17T07:55:53.735152+0000","last_clean":"2024-04-17T07:55:53.735152+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:55:53.735152+0000","last_undegraded":"2024-04-17T07:55:53.735152+0000","last_fullsized":"2024-04-17T07:55:53.735152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779197+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:55:31.779197+0000","last_peered":"2024-04-17T07:55:31.779197+0000","last_clean":"2024-04-17T07:55:31.779197+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:55:31.779197+0000","last_undegraded":"2024-04-17T07:55:31.779197+0000","last_fullsized":"2024-04-17T07:55:31.779197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541086+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:55:32.541086+0000","last_peered":"2024-04-17T07:55:32.541086+0000","last_clean":"2024-04-17T07:55:32.541086+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:55:32.541086+0000","last_undegraded":"2024-04-17T07:55:32.541086+0000","last_fullsized":"2024-04-17T07:55:32.541086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.538944+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:55:32.538944+0000","last_peered":"2024-04-17T07:55:32.538944+0000","last_clean":"2024-04-17T07:55:32.538944+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:55:32.538944+0000","last_undegraded":"2024-04-17T07:55:32.538944+0000","last_fullsized":"2024-04-17T07:55:32.538944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.542864+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:55:32.542864+0000","last_peered":"2024-04-17T07:55:32.542864+0000","last_clean":"2024-04-17T07:55:32.542864+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:55:32.542864+0000","last_undegraded":"2024-04-17T07:55:32.542864+0000","last_fullsized":"2024-04-17T07:55:32.542864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779819+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:55:31.779819+0000","last_peered":"2024-04-17T07:55:31.779819+0000","last_clean":"2024-04-17T07:55:31.779819+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:55:31.779819+0000","last_undegraded":"2024-04-17T07:55:31.779819+0000","last_fullsized":"2024-04-17T07:55:31.779819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:55.535242+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:55:55.535242+0000","last_peered":"2024-04-17T07:55:55.535242+0000","last_clean":"2024-04-17T07:55:55.535242+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:55:55.535242+0000","last_undegraded":"2024-04-17T07:55:55.535242+0000","last_fullsized":"2024-04-17T07:55:55.535242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540563+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:55:32.540563+0000","last_peered":"2024-04-17T07:55:32.540563+0000","last_clean":"2024-04-17T07:55:32.540563+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:55:32.540563+0000","last_undegraded":"2024-04-17T07:55:32.540563+0000","last_fullsized":"2024-04-17T07:55:32.540563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.543021+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:55:32.543021+0000","last_peered":"2024-04-17T07:55:32.543021+0000","last_clean":"2024-04-17T07:55:32.543021+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:55:32.543021+0000","last_undegraded":"2024-04-17T07:55:32.543021+0000","last_fullsized":"2024-04-17T07:55:32.543021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604628+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:55:31.604628+0000","last_peered":"2024-04-17T07:55:31.604628+0000","last_clean":"2024-04-17T07:55:31.604628+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:55:31.604628+0000","last_undegraded":"2024-04-17T07:55:31.604628+0000","last_fullsized":"2024-04-17T07:55:31.604628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540204+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:55:32.540204+0000","last_peered":"2024-04-17T07:55:32.540204+0000","last_clean":"2024-04-17T07:55:32.540204+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:55:32.540204+0000","last_undegraded":"2024-04-17T07:55:32.540204+0000","last_fullsized":"2024-04-17T07:55:32.540204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541542+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:55:32.541542+0000","last_peered":"2024-04-17T07:55:32.541542+0000","last_clean":"2024-04-17T07:55:32.541542+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:55:32.541542+0000","last_undegraded":"2024-04-17T07:55:32.541542+0000","last_fullsized":"2024-04-17T07:55:32.541542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.722269+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:55:52.722269+0000","last_peered":"2024-04-17T07:55:52.722269+0000","last_clean":"2024-04-17T07:55:52.722269+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:55:52.722269+0000","last_undegraded":"2024-04-17T07:55:52.722269+0000","last_fullsized":"2024-04-17T07:55:52.722269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540753+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:55:32.540753+0000","last_peered":"2024-04-17T07:55:32.540753+0000","last_clean":"2024-04-17T07:55:32.540753+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:55:32.540753+0000","last_undegraded":"2024-04-17T07:55:32.540753+0000","last_fullsized":"2024-04-17T07:55:32.540753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.542872+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:55:32.542872+0000","last_peered":"2024-04-17T07:55:32.542872+0000","last_clean":"2024-04-17T07:55:32.542872+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:55:32.542872+0000","last_undegraded":"2024-04-17T07:55:32.542872+0000","last_fullsized":"2024-04-17T07:55:32.542872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'23","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:56.613473+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:55:56.613473+0000","last_peered":"2024-04-17T07:55:56.613473+0000","last_clean":"2024-04-17T07:55:56.613473+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:55:56.613473+0000","last_undegraded":"2024-04-17T07:55:56.613473+0000","last_fullsized":"2024-04-17T07:55:56.613473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":28,"num_write":24,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541435+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:55:32.541435+0000","last_peered":"2024-04-17T07:55:32.541435+0000","last_clean":"2024-04-17T07:55:32.541435+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:55:32.541435+0000","last_undegraded":"2024-04-17T07:55:32.541435+0000","last_fullsized":"2024-04-17T07:55:32.541435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540748+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:55:32.540748+0000","last_peered":"2024-04-17T07:55:32.540748+0000","last_clean":"2024-04-17T07:55:32.540748+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:55:32.540748+0000","last_undegraded":"2024-04-17T07:55:32.540748+0000","last_fullsized":"2024-04-17T07:55:32.540748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.728196+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:55:53.728196+0000","last_peered":"2024-04-17T07:55:53.728196+0000","last_clean":"2024-04-17T07:55:53.728196+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:55:53.728196+0000","last_undegraded":"2024-04-17T07:55:53.728196+0000","last_fullsized":"2024-04-17T07:55:53.728196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.605824+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:55:31.605824+0000","last_peered":"2024-04-17T07:55:31.605824+0000","last_clean":"2024-04-17T07:55:31.605824+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:55:31.605824+0000","last_undegraded":"2024-04-17T07:55:31.605824+0000","last_fullsized":"2024-04-17T07:55:31.605824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539919+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:55:32.539919+0000","last_peered":"2024-04-17T07:55:32.539919+0000","last_clean":"2024-04-17T07:55:32.539919+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:55:32.539919+0000","last_undegraded":"2024-04-17T07:55:32.539919+0000","last_fullsized":"2024-04-17T07:55:32.539919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.722054+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:55:52.722054+0000","last_peered":"2024-04-17T07:55:52.722054+0000","last_clean":"2024-04-17T07:55:52.722054+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:55:52.722054+0000","last_undegraded":"2024-04-17T07:55:52.722054+0000","last_fullsized":"2024-04-17T07:55:52.722054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604524+0000","last_change":"2024-04-17T07:55:26.362163+0000","last_active":"2024-04-17T07:55:31.604524+0000","last_peered":"2024-04-17T07:55:31.604524+0000","last_clean":"2024-04-17T07:55:31.604524+0000","last_became_active":"2024-04-17T07:55:26.361782+0000","last_became_peered":"2024-04-17T07:55:26.361782+0000","last_unstale":"2024-04-17T07:55:31.604524+0000","last_undegraded":"2024-04-17T07:55:31.604524+0000","last_fullsized":"2024-04-17T07:55:31.604524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:56.447221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.543639+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:55:32.543639+0000","last_peered":"2024-04-17T07:55:32.543639+0000","last_clean":"2024-04-17T07:55:32.543639+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:55:32.543639+0000","last_undegraded":"2024-04-17T07:55:32.543639+0000","last_fullsized":"2024-04-17T07:55:32.543639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:55.538589+0000","last_change":"2024-04-17T07:55:25.507054+0000","last_active":"2024-04-17T07:55:55.538589+0000","last_peered":"2024-04-17T07:55:55.538589+0000","last_clean":"2024-04-17T07:55:55.538589+0000","last_became_active":"2024-04-17T07:55:25.506709+0000","last_became_peered":"2024-04-17T07:55:25.506709+0000","last_unstale":"2024-04-17T07:55:55.538589+0000","last_undegraded":"2024-04-17T07:55:55.538589+0000","last_fullsized":"2024-04-17T07:55:55.538589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:53:29.194193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604123+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:55:31.604123+0000","last_peered":"2024-04-17T07:55:31.604123+0000","last_clean":"2024-04-17T07:55:31.604123+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:55:31.604123+0000","last_undegraded":"2024-04-17T07:55:31.604123+0000","last_fullsized":"2024-04-17T07:55:31.604123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607370+0000","last_change":"2024-04-17T07:55:30.611541+0000","last_active":"2024-04-17T07:55:31.607370+0000","last_peered":"2024-04-17T07:55:31.607370+0000","last_clean":"2024-04-17T07:55:31.607370+0000","last_became_active":"2024-04-17T07:55:30.610786+0000","last_became_peered":"2024-04-17T07:55:30.610786+0000","last_unstale":"2024-04-17T07:55:31.607370+0000","last_undegraded":"2024-04-17T07:55:31.607370+0000","last_fullsized":"2024-04-17T07:55:31.607370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:54.266394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.737578+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:55:53.737578+0000","last_peered":"2024-04-17T07:55:53.737578+0000","last_clean":"2024-04-17T07:55:53.737578+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:55:53.737578+0000","last_undegraded":"2024-04-17T07:55:53.737578+0000","last_fullsized":"2024-04-17T07:55:53.737578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539023+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:55:32.539023+0000","last_peered":"2024-04-17T07:55:32.539023+0000","last_clean":"2024-04-17T07:55:32.539023+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:55:32.539023+0000","last_undegraded":"2024-04-17T07:55:32.539023+0000","last_fullsized":"2024-04-17T07:55:32.539023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539064+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:55:32.539064+0000","last_peered":"2024-04-17T07:55:32.539064+0000","last_clean":"2024-04-17T07:55:32.539064+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:55:32.539064+0000","last_undegraded":"2024-04-17T07:55:32.539064+0000","last_fullsized":"2024-04-17T07:55:32.539064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541456+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:55:32.541456+0000","last_peered":"2024-04-17T07:55:32.541456+0000","last_clean":"2024-04-17T07:55:32.541456+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:55:32.541456+0000","last_undegraded":"2024-04-17T07:55:32.541456+0000","last_fullsized":"2024-04-17T07:55:32.541456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539963+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:55:32.539963+0000","last_peered":"2024-04-17T07:55:32.539963+0000","last_clean":"2024-04-17T07:55:32.539963+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:55:32.539963+0000","last_undegraded":"2024-04-17T07:55:32.539963+0000","last_fullsized":"2024-04-17T07:55:32.539963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540991+0000","last_change":"2024-04-17T07:55:30.585288+0000","last_active":"2024-04-17T07:55:32.540991+0000","last_peered":"2024-04-17T07:55:32.540991+0000","last_clean":"2024-04-17T07:55:32.540991+0000","last_became_active":"2024-04-17T07:55:30.584850+0000","last_became_peered":"2024-04-17T07:55:30.584850+0000","last_unstale":"2024-04-17T07:55:32.540991+0000","last_undegraded":"2024-04-17T07:55:32.540991+0000","last_fullsized":"2024-04-17T07:55:32.540991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:46:35.735458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.543619+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:55:32.543619+0000","last_peered":"2024-04-17T07:55:32.543619+0000","last_clean":"2024-04-17T07:55:32.543619+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:55:32.543619+0000","last_undegraded":"2024-04-17T07:55:32.543619+0000","last_fullsized":"2024-04-17T07:55:32.543619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540350+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:55:32.540350+0000","last_peered":"2024-04-17T07:55:32.540350+0000","last_clean":"2024-04-17T07:55:32.540350+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:55:32.540350+0000","last_undegraded":"2024-04-17T07:55:32.540350+0000","last_fullsized":"2024-04-17T07:55:32.540350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607116+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:55:31.607116+0000","last_peered":"2024-04-17T07:55:31.607116+0000","last_clean":"2024-04-17T07:55:31.607116+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:55:31.607116+0000","last_undegraded":"2024-04-17T07:55:31.607116+0000","last_fullsized":"2024-04-17T07:55:31.607116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604224+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:55:31.604224+0000","last_peered":"2024-04-17T07:55:31.604224+0000","last_clean":"2024-04-17T07:55:31.604224+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:55:31.604224+0000","last_undegraded":"2024-04-17T07:55:31.604224+0000","last_fullsized":"2024-04-17T07:55:31.604224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540248+0000","last_change":"2024-04-17T07:55:26.354251+0000","last_active":"2024-04-17T07:55:32.540248+0000","last_peered":"2024-04-17T07:55:32.540248+0000","last_clean":"2024-04-17T07:55:32.540248+0000","last_became_active":"2024-04-17T07:55:26.353856+0000","last_became_peered":"2024-04-17T07:55:26.353856+0000","last_unstale":"2024-04-17T07:55:32.540248+0000","last_undegraded":"2024-04-17T07:55:32.540248+0000","last_fullsized":"2024-04-17T07:55:32.540248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:48:25.782521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606547+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:55:31.606547+0000","last_peered":"2024-04-17T07:55:31.606547+0000","last_clean":"2024-04-17T07:55:31.606547+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:55:31.606547+0000","last_undegraded":"2024-04-17T07:55:31.606547+0000","last_fullsized":"2024-04-17T07:55:31.606547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539010+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:55:32.539010+0000","last_peered":"2024-04-17T07:55:32.539010+0000","last_clean":"2024-04-17T07:55:32.539010+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:55:32.539010+0000","last_undegraded":"2024-04-17T07:55:32.539010+0000","last_fullsized":"2024-04-17T07:55:32.539010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:54.534187+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:55:54.534187+0000","last_peered":"2024-04-17T07:55:54.534187+0000","last_clean":"2024-04-17T07:55:54.534187+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:55:54.534187+0000","last_undegraded":"2024-04-17T07:55:54.534187+0000","last_fullsized":"2024-04-17T07:55:54.534187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604782+0000","last_change":"2024-04-17T07:55:30.606795+0000","last_active":"2024-04-17T07:55:31.604782+0000","last_peered":"2024-04-17T07:55:31.604782+0000","last_clean":"2024-04-17T07:55:31.604782+0000","last_became_active":"2024-04-17T07:55:30.606351+0000","last_became_peered":"2024-04-17T07:55:30.606351+0000","last_unstale":"2024-04-17T07:55:31.604782+0000","last_undegraded":"2024-04-17T07:55:31.604782+0000","last_fullsized":"2024-04-17T07:55:31.604782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:23.481285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779560+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:55:31.779560+0000","last_peered":"2024-04-17T07:55:31.779560+0000","last_clean":"2024-04-17T07:55:31.779560+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:55:31.779560+0000","last_undegraded":"2024-04-17T07:55:31.779560+0000","last_fullsized":"2024-04-17T07:55:31.779560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:54.530224+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:55:54.530224+0000","last_peered":"2024-04-17T07:55:54.530224+0000","last_clean":"2024-04-17T07:55:54.530224+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:55:54.530224+0000","last_undegraded":"2024-04-17T07:55:54.530224+0000","last_fullsized":"2024-04-17T07:55:54.530224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":347,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.603967+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:55:31.603967+0000","last_peered":"2024-04-17T07:55:31.603967+0000","last_clean":"2024-04-17T07:55:31.603967+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:55:31.603967+0000","last_undegraded":"2024-04-17T07:55:31.603967+0000","last_fullsized":"2024-04-17T07:55:31.603967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539640+0000","last_change":"2024-04-17T07:55:30.600998+0000","last_active":"2024-04-17T07:55:32.539640+0000","last_peered":"2024-04-17T07:55:32.539640+0000","last_clean":"2024-04-17T07:55:32.539640+0000","last_became_active":"2024-04-17T07:55:30.600338+0000","last_became_peered":"2024-04-17T07:55:30.600338+0000","last_unstale":"2024-04-17T07:55:32.539640+0000","last_undegraded":"2024-04-17T07:55:32.539640+0000","last_fullsized":"2024-04-17T07:55:32.539640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:47.103983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606170+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:55:31.606170+0000","last_peered":"2024-04-17T07:55:31.606170+0000","last_clean":"2024-04-17T07:55:31.606170+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:55:31.606170+0000","last_undegraded":"2024-04-17T07:55:31.606170+0000","last_fullsized":"2024-04-17T07:55:31.606170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604042+0000","last_change":"2024-04-17T07:55:25.507838+0000","last_active":"2024-04-17T07:55:31.604042+0000","last_peered":"2024-04-17T07:55:31.604042+0000","last_clean":"2024-04-17T07:55:31.604042+0000","last_became_active":"2024-04-17T07:55:25.507101+0000","last_became_peered":"2024-04-17T07:55:25.507101+0000","last_unstale":"2024-04-17T07:55:31.604042+0000","last_undegraded":"2024-04-17T07:55:31.604042+0000","last_fullsized":"2024-04-17T07:55:31.604042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:24.649804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539881+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:55:32.539881+0000","last_peered":"2024-04-17T07:55:32.539881+0000","last_clean":"2024-04-17T07:55:32.539881+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:55:32.539881+0000","last_undegraded":"2024-04-17T07:55:32.539881+0000","last_fullsized":"2024-04-17T07:55:32.539881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541626+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:55:32.541626+0000","last_peered":"2024-04-17T07:55:32.541626+0000","last_clean":"2024-04-17T07:55:32.541626+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:55:32.541626+0000","last_undegraded":"2024-04-17T07:55:32.541626+0000","last_fullsized":"2024-04-17T07:55:32.541626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607478+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:55:31.607478+0000","last_peered":"2024-04-17T07:55:31.607478+0000","last_clean":"2024-04-17T07:55:31.607478+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:55:31.607478+0000","last_undegraded":"2024-04-17T07:55:31.607478+0000","last_fullsized":"2024-04-17T07:55:31.607478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779431+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:55:31.779431+0000","last_peered":"2024-04-17T07:55:31.779431+0000","last_clean":"2024-04-17T07:55:31.779431+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:55:31.779431+0000","last_undegraded":"2024-04-17T07:55:31.779431+0000","last_fullsized":"2024-04-17T07:55:31.779431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.734038+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:55:53.734038+0000","last_peered":"2024-04-17T07:55:53.734038+0000","last_clean":"2024-04-17T07:55:53.734038+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:55:53.734038+0000","last_undegraded":"2024-04-17T07:55:53.734038+0000","last_fullsized":"2024-04-17T07:55:53.734038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780350+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:55:31.780350+0000","last_peered":"2024-04-17T07:55:31.780350+0000","last_clean":"2024-04-17T07:55:31.780350+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:55:31.780350+0000","last_undegraded":"2024-04-17T07:55:31.780350+0000","last_fullsized":"2024-04-17T07:55:31.780350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608676+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:55:31.608676+0000","last_peered":"2024-04-17T07:55:31.608676+0000","last_clean":"2024-04-17T07:55:31.608676+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:55:31.608676+0000","last_undegraded":"2024-04-17T07:55:31.608676+0000","last_fullsized":"2024-04-17T07:55:31.608676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.720331+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:55:52.720331+0000","last_peered":"2024-04-17T07:55:52.720331+0000","last_clean":"2024-04-17T07:55:52.720331+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:55:52.720331+0000","last_undegraded":"2024-04-17T07:55:52.720331+0000","last_fullsized":"2024-04-17T07:55:52.720331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541418+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:55:32.541418+0000","last_peered":"2024-04-17T07:55:32.541418+0000","last_clean":"2024-04-17T07:55:32.541418+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:55:32.541418+0000","last_undegraded":"2024-04-17T07:55:32.541418+0000","last_fullsized":"2024-04-17T07:55:32.541418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541337+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:55:32.541337+0000","last_peered":"2024-04-17T07:55:32.541337+0000","last_clean":"2024-04-17T07:55:32.541337+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:55:32.541337+0000","last_undegraded":"2024-04-17T07:55:32.541337+0000","last_fullsized":"2024-04-17T07:55:32.541337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.723256+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:55:52.723256+0000","last_peered":"2024-04-17T07:55:52.723256+0000","last_clean":"2024-04-17T07:55:52.723256+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:55:52.723256+0000","last_undegraded":"2024-04-17T07:55:52.723256+0000","last_fullsized":"2024-04-17T07:55:52.723256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540686+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:55:32.540686+0000","last_peered":"2024-04-17T07:55:32.540686+0000","last_clean":"2024-04-17T07:55:32.540686+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:55:32.540686+0000","last_undegraded":"2024-04-17T07:55:32.540686+0000","last_fullsized":"2024-04-17T07:55:32.540686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.544257+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:55:32.544257+0000","last_peered":"2024-04-17T07:55:32.544257+0000","last_clean":"2024-04-17T07:55:32.544257+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:55:32.544257+0000","last_undegraded":"2024-04-17T07:55:32.544257+0000","last_fullsized":"2024-04-17T07:55:32.544257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.724264+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:55:52.724264+0000","last_peered":"2024-04-17T07:55:52.724264+0000","last_clean":"2024-04-17T07:55:52.724264+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:55:52.724264+0000","last_undegraded":"2024-04-17T07:55:52.724264+0000","last_fullsized":"2024-04-17T07:55:52.724264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539715+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:55:32.539715+0000","last_peered":"2024-04-17T07:55:32.539715+0000","last_clean":"2024-04-17T07:55:32.539715+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:55:32.539715+0000","last_undegraded":"2024-04-17T07:55:32.539715+0000","last_fullsized":"2024-04-17T07:55:32.539715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541192+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:55:32.541192+0000","last_peered":"2024-04-17T07:55:32.541192+0000","last_clean":"2024-04-17T07:55:32.541192+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:55:32.541192+0000","last_undegraded":"2024-04-17T07:55:32.541192+0000","last_fullsized":"2024-04-17T07:55:32.541192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608630+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:55:31.608630+0000","last_peered":"2024-04-17T07:55:31.608630+0000","last_clean":"2024-04-17T07:55:31.608630+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:55:31.608630+0000","last_undegraded":"2024-04-17T07:55:31.608630+0000","last_fullsized":"2024-04-17T07:55:31.608630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780196+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:55:31.780196+0000","last_peered":"2024-04-17T07:55:31.780196+0000","last_clean":"2024-04-17T07:55:31.780196+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:55:31.780196+0000","last_undegraded":"2024-04-17T07:55:31.780196+0000","last_fullsized":"2024-04-17T07:55:31.780196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541344+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:55:32.541344+0000","last_peered":"2024-04-17T07:55:32.541344+0000","last_clean":"2024-04-17T07:55:32.541344+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:55:32.541344+0000","last_undegraded":"2024-04-17T07:55:32.541344+0000","last_fullsized":"2024-04-17T07:55:32.541344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.720000+0000","last_change":"2024-04-17T07:55:25.507578+0000","last_active":"2024-04-17T07:55:52.720000+0000","last_peered":"2024-04-17T07:55:52.720000+0000","last_clean":"2024-04-17T07:55:52.720000+0000","last_became_active":"2024-04-17T07:55:25.507219+0000","last_became_peered":"2024-04-17T07:55:25.507219+0000","last_unstale":"2024-04-17T07:55:52.720000+0000","last_undegraded":"2024-04-17T07:55:52.720000+0000","last_fullsized":"2024-04-17T07:55:52.720000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:24.445538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604356+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:55:31.604356+0000","last_peered":"2024-04-17T07:55:31.604356+0000","last_clean":"2024-04-17T07:55:31.604356+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:55:31.604356+0000","last_undegraded":"2024-04-17T07:55:31.604356+0000","last_fullsized":"2024-04-17T07:55:31.604356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539899+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:55:32.539899+0000","last_peered":"2024-04-17T07:55:32.539899+0000","last_clean":"2024-04-17T07:55:32.539899+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:55:32.539899+0000","last_undegraded":"2024-04-17T07:55:32.539899+0000","last_fullsized":"2024-04-17T07:55:32.539899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.736779+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:55:53.736779+0000","last_peered":"2024-04-17T07:55:53.736779+0000","last_clean":"2024-04-17T07:55:53.736779+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:55:53.736779+0000","last_undegraded":"2024-04-17T07:55:53.736779+0000","last_fullsized":"2024-04-17T07:55:53.736779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604849+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:55:31.604849+0000","last_peered":"2024-04-17T07:55:31.604849+0000","last_clean":"2024-04-17T07:55:31.604849+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:55:31.604849+0000","last_undegraded":"2024-04-17T07:55:31.604849+0000","last_fullsized":"2024-04-17T07:55:31.604849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540170+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:55:32.540170+0000","last_peered":"2024-04-17T07:55:32.540170+0000","last_clean":"2024-04-17T07:55:32.540170+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:55:32.540170+0000","last_undegraded":"2024-04-17T07:55:32.540170+0000","last_fullsized":"2024-04-17T07:55:32.540170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540423+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:55:32.540423+0000","last_peered":"2024-04-17T07:55:32.540423+0000","last_clean":"2024-04-17T07:55:32.540423+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:55:32.540423+0000","last_undegraded":"2024-04-17T07:55:32.540423+0000","last_fullsized":"2024-04-17T07:55:32.540423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604544+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:55:31.604544+0000","last_peered":"2024-04-17T07:55:31.604544+0000","last_clean":"2024-04-17T07:55:31.604544+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:55:31.604544+0000","last_undegraded":"2024-04-17T07:55:31.604544+0000","last_fullsized":"2024-04-17T07:55:31.604544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.538743+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:55:32.538743+0000","last_peered":"2024-04-17T07:55:32.538743+0000","last_clean":"2024-04-17T07:55:32.538743+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:55:32.538743+0000","last_undegraded":"2024-04-17T07:55:32.538743+0000","last_fullsized":"2024-04-17T07:55:32.538743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607279+0000","last_change":"2024-04-17T07:55:25.530309+0000","last_active":"2024-04-17T07:55:31.607279+0000","last_peered":"2024-04-17T07:55:31.607279+0000","last_clean":"2024-04-17T07:55:31.607279+0000","last_became_active":"2024-04-17T07:55:25.529880+0000","last_became_peered":"2024-04-17T07:55:25.529880+0000","last_unstale":"2024-04-17T07:55:31.607279+0000","last_undegraded":"2024-04-17T07:55:31.607279+0000","last_fullsized":"2024-04-17T07:55:31.607279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:00.265792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604799+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:55:31.604799+0000","last_peered":"2024-04-17T07:55:31.604799+0000","last_clean":"2024-04-17T07:55:31.604799+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:55:31.604799+0000","last_undegraded":"2024-04-17T07:55:31.604799+0000","last_fullsized":"2024-04-17T07:55:31.604799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540341+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:55:32.540341+0000","last_peered":"2024-04-17T07:55:32.540341+0000","last_clean":"2024-04-17T07:55:32.540341+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:55:32.540341+0000","last_undegraded":"2024-04-17T07:55:32.540341+0000","last_fullsized":"2024-04-17T07:55:32.540341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608749+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:55:31.608749+0000","last_peered":"2024-04-17T07:55:31.608749+0000","last_clean":"2024-04-17T07:55:31.608749+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:55:31.608749+0000","last_undegraded":"2024-04-17T07:55:31.608749+0000","last_fullsized":"2024-04-17T07:55:31.608749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608521+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:55:31.608521+0000","last_peered":"2024-04-17T07:55:31.608521+0000","last_clean":"2024-04-17T07:55:31.608521+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:55:31.608521+0000","last_undegraded":"2024-04-17T07:55:31.608521+0000","last_fullsized":"2024-04-17T07:55:31.608521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540969+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:55:32.540969+0000","last_peered":"2024-04-17T07:55:32.540969+0000","last_clean":"2024-04-17T07:55:32.540969+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:55:32.540969+0000","last_undegraded":"2024-04-17T07:55:32.540969+0000","last_fullsized":"2024-04-17T07:55:32.540969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607386+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:55:31.607386+0000","last_peered":"2024-04-17T07:55:31.607386+0000","last_clean":"2024-04-17T07:55:31.607386+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:55:31.607386+0000","last_undegraded":"2024-04-17T07:55:31.607386+0000","last_fullsized":"2024-04-17T07:55:31.607386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540151+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:55:32.540151+0000","last_peered":"2024-04-17T07:55:32.540151+0000","last_clean":"2024-04-17T07:55:32.540151+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:55:32.540151+0000","last_undegraded":"2024-04-17T07:55:32.540151+0000","last_fullsized":"2024-04-17T07:55:32.540151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609264+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:55:31.609264+0000","last_peered":"2024-04-17T07:55:31.609264+0000","last_clean":"2024-04-17T07:55:31.609264+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:55:31.609264+0000","last_undegraded":"2024-04-17T07:55:31.609264+0000","last_fullsized":"2024-04-17T07:55:31.609264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780721+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:55:31.780721+0000","last_peered":"2024-04-17T07:55:31.780721+0000","last_clean":"2024-04-17T07:55:31.780721+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:55:31.780721+0000","last_undegraded":"2024-04-17T07:55:31.780721+0000","last_fullsized":"2024-04-17T07:55:31.780721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.538936+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:55:32.538936+0000","last_peered":"2024-04-17T07:55:32.538936+0000","last_clean":"2024-04-17T07:55:32.538936+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:55:32.538936+0000","last_undegraded":"2024-04-17T07:55:32.538936+0000","last_fullsized":"2024-04-17T07:55:32.538936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779608+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:55:31.779608+0000","last_peered":"2024-04-17T07:55:31.779608+0000","last_clean":"2024-04-17T07:55:31.779608+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:55:31.779608+0000","last_undegraded":"2024-04-17T07:55:31.779608+0000","last_fullsized":"2024-04-17T07:55:31.779608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540661+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:55:32.540661+0000","last_peered":"2024-04-17T07:55:32.540661+0000","last_clean":"2024-04-17T07:55:32.540661+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:55:32.540661+0000","last_undegraded":"2024-04-17T07:55:32.540661+0000","last_fullsized":"2024-04-17T07:55:32.540661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606729+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:55:31.606729+0000","last_peered":"2024-04-17T07:55:31.606729+0000","last_clean":"2024-04-17T07:55:31.606729+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:55:31.606729+0000","last_undegraded":"2024-04-17T07:55:31.606729+0000","last_fullsized":"2024-04-17T07:55:31.606729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540136+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:55:32.540136+0000","last_peered":"2024-04-17T07:55:32.540136+0000","last_clean":"2024-04-17T07:55:32.540136+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:55:32.540136+0000","last_undegraded":"2024-04-17T07:55:32.540136+0000","last_fullsized":"2024-04-17T07:55:32.540136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46298,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":34,"num_read_kb":48,"num_write":135,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":118047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383701,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":481893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645337,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1092693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808862,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070500,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":477841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":487657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":478693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":520862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":477319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1609728,"data_stored":1073546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":43647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":43875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:55:58.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:55:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 -- 172.21.15.46:0/2628517015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1085e0 msgr2=0x7f552c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 --2- 172.21.15.46:0/2628517015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1085e0 0x7f552c108a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5520009860 tx=0x7f552002bb70 comp rx=0 tx=0).stop 2024-04-17T07:55:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 -- 172.21.15.46:0/2628517015 shutdown_connections 2024-04-17T07:55:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 --2- 172.21.15.46:0/2628517015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 0x7f552c110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 --2- 172.21.15.46:0/2628517015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f552c1097d0 0x7f552c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 --2- 172.21.15.46:0/2628517015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1085e0 0x7f552c108a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 -- 172.21.15.46:0/2628517015 >> 172.21.15.46:0/2628517015 conn(0x7f552c0775b0 msgr2=0x7f552c0799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.887+0000 7f5534400640 1 -- 172.21.15.46:0/2628517015 shutdown_connections 2024-04-17T07:55:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.888+0000 7f5534400640 1 -- 172.21.15.46:0/2628517015 wait complete. 2024-04-17T07:55:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 Processor -- start 2024-04-17T07:55:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 -- start start 2024-04-17T07:55:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f552c1085e0 0x7f552c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1097d0 0x7f552c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 0x7f552c1a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f552c114230 con 0x7f552c10a190 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1140b0 con 0x7f552c1097d0 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.889+0000 7f5534400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1143b0 con 0x7f552c1085e0 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5532400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 0x7f552c1a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f552c1085e0 0x7f552c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5532400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 0x7f552c1a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35888/0 (socket says 172.21.15.46:35888) 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5532400640 1 -- 172.21.15.46:0/1556849907 learned_addr learned my addr 172.21.15.46:0/1556849907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531000640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1097d0 0x7f552c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531a00640 1 -- 172.21.15.46:0/1556849907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1097d0 msgr2=0x7f552c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531a00640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1097d0 0x7f552c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531a00640 1 -- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 msgr2=0x7f552c1a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531a00640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 0x7f552c1a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531a00640 1 -- 172.21.15.46:0/1556849907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c1a6540 con 0x7f552c1085e0 2024-04-17T07:55:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5532400640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 0x7f552c1a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:55:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.890+0000 7f5531a00640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f552c1085e0 0x7f552c19e920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5520009780 tx=0x7f5520032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.891+0000 7f552a000640 1 -- 172.21.15.46:0/1556849907 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520041020 con 0x7f552c1085e0 2024-04-17T07:55:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.891+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c1a67d0 con 0x7f552c1085e0 2024-04-17T07:55:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.891+0000 7f552a000640 1 -- 172.21.15.46:0/1556849907 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f552002c9f0 con 0x7f552c1085e0 2024-04-17T07:55:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.891+0000 7f552a000640 1 -- 172.21.15.46:0/1556849907 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520039040 con 0x7f552c1085e0 2024-04-17T07:55:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.891+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a6ce0 con 0x7f552c1085e0 2024-04-17T07:55:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.892+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c10f9f0 con 0x7f552c1085e0 2024-04-17T07:55:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.894+0000 7f552a000640 1 -- 172.21.15.46:0/1556849907 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5520044020 con 0x7f552c1085e0 2024-04-17T07:55:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.896+0000 7f552a000640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f55140a3c40 0x7f55140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:55:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.896+0000 7f552a000640 1 -- 172.21.15.46:0/1556849907 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f55200f1f10 con 0x7f552c1085e0 2024-04-17T07:55:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.896+0000 7f5531000640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f55140a3c40 0x7f55140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:55:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.897+0000 7f5531000640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f55140a3c40 0x7f55140a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f552c19fe30 tx=0x7f5518005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:55:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:58.900+0000 7f552a000640 1 -- 172.21.15.46:0/1556849907 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f552004b020 con 0x7f552c1085e0 2024-04-17T07:55:59.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:58 smithi062 ceph-mon[36507]: pgmap v337: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-17T07:55:59.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:58 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3018536782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:59.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.100+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f552c0623a0 con 0x7f55140a3c40 2024-04-17T07:55:59.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:58 smithi046 ceph-mon[32038]: pgmap v337: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-17T07:55:59.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:58 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3018536782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:59.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.118+0000 7f552a000640 1 -- 172.21.15.46:0/1556849907 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f552c0623a0 con 0x7f55140a3c40 2024-04-17T07:55:59.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:55:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f55140a3c40 msgr2=0x7f55140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f55140a3c40 0x7f55140a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f552c19fe30 tx=0x7f5518005f90 comp rx=0 tx=0).stop 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f552c1085e0 msgr2=0x7f552c19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f552c1085e0 0x7f552c19e920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5520009780 tx=0x7f5520032430 comp rx=0 tx=0).stop 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 shutdown_connections 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f55140a3c40 0x7f55140a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f552c10a190 0x7f552c1a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f552c1097d0 0x7f552c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 --2- 172.21.15.46:0/1556849907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f552c1085e0 0x7f552c19e920 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.126+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 >> 172.21.15.46:0/1556849907 conn(0x7f552c0775b0 msgr2=0x7f552c10cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.127+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 shutdown_connections 2024-04-17T07:55:59.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:55:59.127+0000 7f5534400640 1 -- 172.21.15.46:0/1556849907 wait complete. 2024-04-17T07:55:59.145 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-17T07:55:57.514390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636666,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":160,"num_read_kb":157,"num_write":316,"num_write_kb":3025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":255,"ondisk_log_size":255,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375196,"kb_used_data":13548,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566628,"statfs":{"total":1151940427776,"available":1151556227072,"internally_reserved":0,"allocated":13873152,"data_stored":7592257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43439,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":34,"num_read_kb":48,"num_write":89,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005977"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540145+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:55:32.540145+0000","last_peered":"2024-04-17T07:55:32.540145+0000","last_clean":"2024-04-17T07:55:32.540145+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:55:32.540145+0000","last_undegraded":"2024-04-17T07:55:32.540145+0000","last_fullsized":"2024-04-17T07:55:32.540145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541318+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:55:32.541318+0000","last_peered":"2024-04-17T07:55:32.541318+0000","last_clean":"2024-04-17T07:55:32.541318+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:55:32.541318+0000","last_undegraded":"2024-04-17T07:55:32.541318+0000","last_fullsized":"2024-04-17T07:55:32.541318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540623+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:55:32.540623+0000","last_peered":"2024-04-17T07:55:32.540623+0000","last_clean":"2024-04-17T07:55:32.540623+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:55:32.540623+0000","last_undegraded":"2024-04-17T07:55:32.540623+0000","last_fullsized":"2024-04-17T07:55:32.540623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541526+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:55:32.541526+0000","last_peered":"2024-04-17T07:55:32.541526+0000","last_clean":"2024-04-17T07:55:32.541526+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:55:32.541526+0000","last_undegraded":"2024-04-17T07:55:32.541526+0000","last_fullsized":"2024-04-17T07:55:32.541526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540948+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:55:32.540948+0000","last_peered":"2024-04-17T07:55:32.540948+0000","last_clean":"2024-04-17T07:55:32.540948+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:55:32.540948+0000","last_undegraded":"2024-04-17T07:55:32.540948+0000","last_fullsized":"2024-04-17T07:55:32.540948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540735+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:55:32.540735+0000","last_peered":"2024-04-17T07:55:32.540735+0000","last_clean":"2024-04-17T07:55:32.540735+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:55:32.540735+0000","last_undegraded":"2024-04-17T07:55:32.540735+0000","last_fullsized":"2024-04-17T07:55:32.540735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604045+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:55:31.604045+0000","last_peered":"2024-04-17T07:55:31.604045+0000","last_clean":"2024-04-17T07:55:31.604045+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:55:31.604045+0000","last_undegraded":"2024-04-17T07:55:31.604045+0000","last_fullsized":"2024-04-17T07:55:31.604045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779511+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:55:31.779511+0000","last_peered":"2024-04-17T07:55:31.779511+0000","last_clean":"2024-04-17T07:55:31.779511+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:55:31.779511+0000","last_undegraded":"2024-04-17T07:55:31.779511+0000","last_fullsized":"2024-04-17T07:55:31.779511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540539+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:55:32.540539+0000","last_peered":"2024-04-17T07:55:32.540539+0000","last_clean":"2024-04-17T07:55:32.540539+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:55:32.540539+0000","last_undegraded":"2024-04-17T07:55:32.540539+0000","last_fullsized":"2024-04-17T07:55:32.540539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541050+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:55:32.541050+0000","last_peered":"2024-04-17T07:55:32.541050+0000","last_clean":"2024-04-17T07:55:32.541050+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:55:32.541050+0000","last_undegraded":"2024-04-17T07:55:32.541050+0000","last_fullsized":"2024-04-17T07:55:32.541050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604781+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:55:31.604781+0000","last_peered":"2024-04-17T07:55:31.604781+0000","last_clean":"2024-04-17T07:55:31.604781+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:55:31.604781+0000","last_undegraded":"2024-04-17T07:55:31.604781+0000","last_fullsized":"2024-04-17T07:55:31.604781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541330+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:55:32.541330+0000","last_peered":"2024-04-17T07:55:32.541330+0000","last_clean":"2024-04-17T07:55:32.541330+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:55:32.541330+0000","last_undegraded":"2024-04-17T07:55:32.541330+0000","last_fullsized":"2024-04-17T07:55:32.541330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606373+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:55:31.606373+0000","last_peered":"2024-04-17T07:55:31.606373+0000","last_clean":"2024-04-17T07:55:31.606373+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:55:31.606373+0000","last_undegraded":"2024-04-17T07:55:31.606373+0000","last_fullsized":"2024-04-17T07:55:31.606373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539861+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:55:32.539861+0000","last_peered":"2024-04-17T07:55:32.539861+0000","last_clean":"2024-04-17T07:55:32.539861+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:55:32.539861+0000","last_undegraded":"2024-04-17T07:55:32.539861+0000","last_fullsized":"2024-04-17T07:55:32.539861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540270+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:55:32.540270+0000","last_peered":"2024-04-17T07:55:32.540270+0000","last_clean":"2024-04-17T07:55:32.540270+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:55:32.540270+0000","last_undegraded":"2024-04-17T07:55:32.540270+0000","last_fullsized":"2024-04-17T07:55:32.540270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539579+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:55:32.539579+0000","last_peered":"2024-04-17T07:55:32.539579+0000","last_clean":"2024-04-17T07:55:32.539579+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:55:32.539579+0000","last_undegraded":"2024-04-17T07:55:32.539579+0000","last_fullsized":"2024-04-17T07:55:32.539579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540015+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:55:32.540015+0000","last_peered":"2024-04-17T07:55:32.540015+0000","last_clean":"2024-04-17T07:55:32.540015+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:55:32.540015+0000","last_undegraded":"2024-04-17T07:55:32.540015+0000","last_fullsized":"2024-04-17T07:55:32.540015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.544098+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:55:32.544098+0000","last_peered":"2024-04-17T07:55:32.544098+0000","last_clean":"2024-04-17T07:55:32.544098+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:55:32.544098+0000","last_undegraded":"2024-04-17T07:55:32.544098+0000","last_fullsized":"2024-04-17T07:55:32.544098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609114+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:55:31.609114+0000","last_peered":"2024-04-17T07:55:31.609114+0000","last_clean":"2024-04-17T07:55:31.609114+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:55:31.609114+0000","last_undegraded":"2024-04-17T07:55:31.609114+0000","last_fullsized":"2024-04-17T07:55:31.609114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608992+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:55:31.608992+0000","last_peered":"2024-04-17T07:55:31.608992+0000","last_clean":"2024-04-17T07:55:31.608992+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:55:31.608992+0000","last_undegraded":"2024-04-17T07:55:31.608992+0000","last_fullsized":"2024-04-17T07:55:31.608992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.740250+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:55:53.740250+0000","last_peered":"2024-04-17T07:55:53.740250+0000","last_clean":"2024-04-17T07:55:53.740250+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:55:53.740250+0000","last_undegraded":"2024-04-17T07:55:53.740250+0000","last_fullsized":"2024-04-17T07:55:53.740250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780610+0000","last_change":"2024-04-17T07:55:26.373262+0000","last_active":"2024-04-17T07:55:31.780610+0000","last_peered":"2024-04-17T07:55:31.780610+0000","last_clean":"2024-04-17T07:55:31.780610+0000","last_became_active":"2024-04-17T07:55:26.366248+0000","last_became_peered":"2024-04-17T07:55:26.366248+0000","last_unstale":"2024-04-17T07:55:31.780610+0000","last_undegraded":"2024-04-17T07:55:31.780610+0000","last_fullsized":"2024-04-17T07:55:31.780610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:16.251444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540058+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:55:32.540058+0000","last_peered":"2024-04-17T07:55:32.540058+0000","last_clean":"2024-04-17T07:55:32.540058+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:55:32.540058+0000","last_undegraded":"2024-04-17T07:55:32.540058+0000","last_fullsized":"2024-04-17T07:55:32.540058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.781038+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:55:31.781038+0000","last_peered":"2024-04-17T07:55:31.781038+0000","last_clean":"2024-04-17T07:55:31.781038+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:55:31.781038+0000","last_undegraded":"2024-04-17T07:55:31.781038+0000","last_fullsized":"2024-04-17T07:55:31.781038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540314+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:55:32.540314+0000","last_peered":"2024-04-17T07:55:32.540314+0000","last_clean":"2024-04-17T07:55:32.540314+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:55:32.540314+0000","last_undegraded":"2024-04-17T07:55:32.540314+0000","last_fullsized":"2024-04-17T07:55:32.540314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541095+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:55:32.541095+0000","last_peered":"2024-04-17T07:55:32.541095+0000","last_clean":"2024-04-17T07:55:32.541095+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:55:32.541095+0000","last_undegraded":"2024-04-17T07:55:32.541095+0000","last_fullsized":"2024-04-17T07:55:32.541095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.728146+0000","last_change":"2024-04-17T07:55:25.509157+0000","last_active":"2024-04-17T07:55:53.728146+0000","last_peered":"2024-04-17T07:55:53.728146+0000","last_clean":"2024-04-17T07:55:53.728146+0000","last_became_active":"2024-04-17T07:55:25.508793+0000","last_became_peered":"2024-04-17T07:55:25.508793+0000","last_unstale":"2024-04-17T07:55:53.728146+0000","last_undegraded":"2024-04-17T07:55:53.728146+0000","last_fullsized":"2024-04-17T07:55:53.728146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:55:29.596338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540541+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:55:32.540541+0000","last_peered":"2024-04-17T07:55:32.540541+0000","last_clean":"2024-04-17T07:55:32.540541+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:55:32.540541+0000","last_undegraded":"2024-04-17T07:55:32.540541+0000","last_fullsized":"2024-04-17T07:55:32.540541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539773+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:55:32.539773+0000","last_peered":"2024-04-17T07:55:32.539773+0000","last_clean":"2024-04-17T07:55:32.539773+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:55:32.539773+0000","last_undegraded":"2024-04-17T07:55:32.539773+0000","last_fullsized":"2024-04-17T07:55:32.539773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541148+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:55:32.541148+0000","last_peered":"2024-04-17T07:55:32.541148+0000","last_clean":"2024-04-17T07:55:32.541148+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:55:32.541148+0000","last_undegraded":"2024-04-17T07:55:32.541148+0000","last_fullsized":"2024-04-17T07:55:32.541148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540099+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:55:32.540099+0000","last_peered":"2024-04-17T07:55:32.540099+0000","last_clean":"2024-04-17T07:55:32.540099+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:55:32.540099+0000","last_undegraded":"2024-04-17T07:55:32.540099+0000","last_fullsized":"2024-04-17T07:55:32.540099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609019+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:55:31.609019+0000","last_peered":"2024-04-17T07:55:31.609019+0000","last_clean":"2024-04-17T07:55:31.609019+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:55:31.609019+0000","last_undegraded":"2024-04-17T07:55:31.609019+0000","last_fullsized":"2024-04-17T07:55:31.609019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:55.535908+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:55:55.535908+0000","last_peered":"2024-04-17T07:55:55.535908+0000","last_clean":"2024-04-17T07:55:55.535908+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:55:55.535908+0000","last_undegraded":"2024-04-17T07:55:55.535908+0000","last_fullsized":"2024-04-17T07:55:55.535908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540874+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:55:32.540874+0000","last_peered":"2024-04-17T07:55:32.540874+0000","last_clean":"2024-04-17T07:55:32.540874+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:55:32.540874+0000","last_undegraded":"2024-04-17T07:55:32.540874+0000","last_fullsized":"2024-04-17T07:55:32.540874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609373+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:55:31.609373+0000","last_peered":"2024-04-17T07:55:31.609373+0000","last_clean":"2024-04-17T07:55:31.609373+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:55:31.609373+0000","last_undegraded":"2024-04-17T07:55:31.609373+0000","last_fullsized":"2024-04-17T07:55:31.609373+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.730259+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:55:53.730259+0000","last_peered":"2024-04-17T07:55:53.730259+0000","last_clean":"2024-04-17T07:55:53.730259+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:55:53.730259+0000","last_undegraded":"2024-04-17T07:55:53.730259+0000","last_fullsized":"2024-04-17T07:55:53.730259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539842+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:55:32.539842+0000","last_peered":"2024-04-17T07:55:32.539842+0000","last_clean":"2024-04-17T07:55:32.539842+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:55:32.539842+0000","last_undegraded":"2024-04-17T07:55:32.539842+0000","last_fullsized":"2024-04-17T07:55:32.539842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540068+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:55:32.540068+0000","last_peered":"2024-04-17T07:55:32.540068+0000","last_clean":"2024-04-17T07:55:32.540068+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:55:32.540068+0000","last_undegraded":"2024-04-17T07:55:32.540068+0000","last_fullsized":"2024-04-17T07:55:32.540068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.735152+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:55:53.735152+0000","last_peered":"2024-04-17T07:55:53.735152+0000","last_clean":"2024-04-17T07:55:53.735152+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:55:53.735152+0000","last_undegraded":"2024-04-17T07:55:53.735152+0000","last_fullsized":"2024-04-17T07:55:53.735152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779197+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:55:31.779197+0000","last_peered":"2024-04-17T07:55:31.779197+0000","last_clean":"2024-04-17T07:55:31.779197+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:55:31.779197+0000","last_undegraded":"2024-04-17T07:55:31.779197+0000","last_fullsized":"2024-04-17T07:55:31.779197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541086+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:55:32.541086+0000","last_peered":"2024-04-17T07:55:32.541086+0000","last_clean":"2024-04-17T07:55:32.541086+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:55:32.541086+0000","last_undegraded":"2024-04-17T07:55:32.541086+0000","last_fullsized":"2024-04-17T07:55:32.541086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.538944+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:55:32.538944+0000","last_peered":"2024-04-17T07:55:32.538944+0000","last_clean":"2024-04-17T07:55:32.538944+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:55:32.538944+0000","last_undegraded":"2024-04-17T07:55:32.538944+0000","last_fullsized":"2024-04-17T07:55:32.538944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.542864+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:55:32.542864+0000","last_peered":"2024-04-17T07:55:32.542864+0000","last_clean":"2024-04-17T07:55:32.542864+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:55:32.542864+0000","last_undegraded":"2024-04-17T07:55:32.542864+0000","last_fullsized":"2024-04-17T07:55:32.542864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779819+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:55:31.779819+0000","last_peered":"2024-04-17T07:55:31.779819+0000","last_clean":"2024-04-17T07:55:31.779819+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:55:31.779819+0000","last_undegraded":"2024-04-17T07:55:31.779819+0000","last_fullsized":"2024-04-17T07:55:31.779819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:55.535242+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:55:55.535242+0000","last_peered":"2024-04-17T07:55:55.535242+0000","last_clean":"2024-04-17T07:55:55.535242+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:55:55.535242+0000","last_undegraded":"2024-04-17T07:55:55.535242+0000","last_fullsized":"2024-04-17T07:55:55.535242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540563+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:55:32.540563+0000","last_peered":"2024-04-17T07:55:32.540563+0000","last_clean":"2024-04-17T07:55:32.540563+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:55:32.540563+0000","last_undegraded":"2024-04-17T07:55:32.540563+0000","last_fullsized":"2024-04-17T07:55:32.540563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.543021+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:55:32.543021+0000","last_peered":"2024-04-17T07:55:32.543021+0000","last_clean":"2024-04-17T07:55:32.543021+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:55:32.543021+0000","last_undegraded":"2024-04-17T07:55:32.543021+0000","last_fullsized":"2024-04-17T07:55:32.543021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604628+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:55:31.604628+0000","last_peered":"2024-04-17T07:55:31.604628+0000","last_clean":"2024-04-17T07:55:31.604628+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:55:31.604628+0000","last_undegraded":"2024-04-17T07:55:31.604628+0000","last_fullsized":"2024-04-17T07:55:31.604628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540204+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:55:32.540204+0000","last_peered":"2024-04-17T07:55:32.540204+0000","last_clean":"2024-04-17T07:55:32.540204+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:55:32.540204+0000","last_undegraded":"2024-04-17T07:55:32.540204+0000","last_fullsized":"2024-04-17T07:55:32.540204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541542+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:55:32.541542+0000","last_peered":"2024-04-17T07:55:32.541542+0000","last_clean":"2024-04-17T07:55:32.541542+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:55:32.541542+0000","last_undegraded":"2024-04-17T07:55:32.541542+0000","last_fullsized":"2024-04-17T07:55:32.541542+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.722269+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:55:52.722269+0000","last_peered":"2024-04-17T07:55:52.722269+0000","last_clean":"2024-04-17T07:55:52.722269+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:55:52.722269+0000","last_undegraded":"2024-04-17T07:55:52.722269+0000","last_fullsized":"2024-04-17T07:55:52.722269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540753+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:55:32.540753+0000","last_peered":"2024-04-17T07:55:32.540753+0000","last_clean":"2024-04-17T07:55:32.540753+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:55:32.540753+0000","last_undegraded":"2024-04-17T07:55:32.540753+0000","last_fullsized":"2024-04-17T07:55:32.540753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.542872+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:55:32.542872+0000","last_peered":"2024-04-17T07:55:32.542872+0000","last_clean":"2024-04-17T07:55:32.542872+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:55:32.542872+0000","last_undegraded":"2024-04-17T07:55:32.542872+0000","last_fullsized":"2024-04-17T07:55:32.542872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'23","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:56.613473+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:55:56.613473+0000","last_peered":"2024-04-17T07:55:56.613473+0000","last_clean":"2024-04-17T07:55:56.613473+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:55:56.613473+0000","last_undegraded":"2024-04-17T07:55:56.613473+0000","last_fullsized":"2024-04-17T07:55:56.613473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":28,"num_write":24,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541435+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:55:32.541435+0000","last_peered":"2024-04-17T07:55:32.541435+0000","last_clean":"2024-04-17T07:55:32.541435+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:55:32.541435+0000","last_undegraded":"2024-04-17T07:55:32.541435+0000","last_fullsized":"2024-04-17T07:55:32.541435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540748+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:55:32.540748+0000","last_peered":"2024-04-17T07:55:32.540748+0000","last_clean":"2024-04-17T07:55:32.540748+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:55:32.540748+0000","last_undegraded":"2024-04-17T07:55:32.540748+0000","last_fullsized":"2024-04-17T07:55:32.540748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.728196+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:55:53.728196+0000","last_peered":"2024-04-17T07:55:53.728196+0000","last_clean":"2024-04-17T07:55:53.728196+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:55:53.728196+0000","last_undegraded":"2024-04-17T07:55:53.728196+0000","last_fullsized":"2024-04-17T07:55:53.728196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.605824+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:55:31.605824+0000","last_peered":"2024-04-17T07:55:31.605824+0000","last_clean":"2024-04-17T07:55:31.605824+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:55:31.605824+0000","last_undegraded":"2024-04-17T07:55:31.605824+0000","last_fullsized":"2024-04-17T07:55:31.605824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539919+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:55:32.539919+0000","last_peered":"2024-04-17T07:55:32.539919+0000","last_clean":"2024-04-17T07:55:32.539919+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:55:32.539919+0000","last_undegraded":"2024-04-17T07:55:32.539919+0000","last_fullsized":"2024-04-17T07:55:32.539919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.722054+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:55:52.722054+0000","last_peered":"2024-04-17T07:55:52.722054+0000","last_clean":"2024-04-17T07:55:52.722054+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:55:52.722054+0000","last_undegraded":"2024-04-17T07:55:52.722054+0000","last_fullsized":"2024-04-17T07:55:52.722054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604524+0000","last_change":"2024-04-17T07:55:26.362163+0000","last_active":"2024-04-17T07:55:31.604524+0000","last_peered":"2024-04-17T07:55:31.604524+0000","last_clean":"2024-04-17T07:55:31.604524+0000","last_became_active":"2024-04-17T07:55:26.361782+0000","last_became_peered":"2024-04-17T07:55:26.361782+0000","last_unstale":"2024-04-17T07:55:31.604524+0000","last_undegraded":"2024-04-17T07:55:31.604524+0000","last_fullsized":"2024-04-17T07:55:31.604524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:56.447221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.543639+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:55:32.543639+0000","last_peered":"2024-04-17T07:55:32.543639+0000","last_clean":"2024-04-17T07:55:32.543639+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:55:32.543639+0000","last_undegraded":"2024-04-17T07:55:32.543639+0000","last_fullsized":"2024-04-17T07:55:32.543639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:55.538589+0000","last_change":"2024-04-17T07:55:25.507054+0000","last_active":"2024-04-17T07:55:55.538589+0000","last_peered":"2024-04-17T07:55:55.538589+0000","last_clean":"2024-04-17T07:55:55.538589+0000","last_became_active":"2024-04-17T07:55:25.506709+0000","last_became_peered":"2024-04-17T07:55:25.506709+0000","last_unstale":"2024-04-17T07:55:55.538589+0000","last_undegraded":"2024-04-17T07:55:55.538589+0000","last_fullsized":"2024-04-17T07:55:55.538589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:53:29.194193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604123+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:55:31.604123+0000","last_peered":"2024-04-17T07:55:31.604123+0000","last_clean":"2024-04-17T07:55:31.604123+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:55:31.604123+0000","last_undegraded":"2024-04-17T07:55:31.604123+0000","last_fullsized":"2024-04-17T07:55:31.604123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607370+0000","last_change":"2024-04-17T07:55:30.611541+0000","last_active":"2024-04-17T07:55:31.607370+0000","last_peered":"2024-04-17T07:55:31.607370+0000","last_clean":"2024-04-17T07:55:31.607370+0000","last_became_active":"2024-04-17T07:55:30.610786+0000","last_became_peered":"2024-04-17T07:55:30.610786+0000","last_unstale":"2024-04-17T07:55:31.607370+0000","last_undegraded":"2024-04-17T07:55:31.607370+0000","last_fullsized":"2024-04-17T07:55:31.607370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:54.266394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.737578+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:55:53.737578+0000","last_peered":"2024-04-17T07:55:53.737578+0000","last_clean":"2024-04-17T07:55:53.737578+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:55:53.737578+0000","last_undegraded":"2024-04-17T07:55:53.737578+0000","last_fullsized":"2024-04-17T07:55:53.737578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539023+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:55:32.539023+0000","last_peered":"2024-04-17T07:55:32.539023+0000","last_clean":"2024-04-17T07:55:32.539023+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:55:32.539023+0000","last_undegraded":"2024-04-17T07:55:32.539023+0000","last_fullsized":"2024-04-17T07:55:32.539023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539064+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:55:32.539064+0000","last_peered":"2024-04-17T07:55:32.539064+0000","last_clean":"2024-04-17T07:55:32.539064+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:55:32.539064+0000","last_undegraded":"2024-04-17T07:55:32.539064+0000","last_fullsized":"2024-04-17T07:55:32.539064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541456+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:55:32.541456+0000","last_peered":"2024-04-17T07:55:32.541456+0000","last_clean":"2024-04-17T07:55:32.541456+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:55:32.541456+0000","last_undegraded":"2024-04-17T07:55:32.541456+0000","last_fullsized":"2024-04-17T07:55:32.541456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539963+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:55:32.539963+0000","last_peered":"2024-04-17T07:55:32.539963+0000","last_clean":"2024-04-17T07:55:32.539963+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:55:32.539963+0000","last_undegraded":"2024-04-17T07:55:32.539963+0000","last_fullsized":"2024-04-17T07:55:32.539963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540991+0000","last_change":"2024-04-17T07:55:30.585288+0000","last_active":"2024-04-17T07:55:32.540991+0000","last_peered":"2024-04-17T07:55:32.540991+0000","last_clean":"2024-04-17T07:55:32.540991+0000","last_became_active":"2024-04-17T07:55:30.584850+0000","last_became_peered":"2024-04-17T07:55:30.584850+0000","last_unstale":"2024-04-17T07:55:32.540991+0000","last_undegraded":"2024-04-17T07:55:32.540991+0000","last_fullsized":"2024-04-17T07:55:32.540991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:46:35.735458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.543619+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:55:32.543619+0000","last_peered":"2024-04-17T07:55:32.543619+0000","last_clean":"2024-04-17T07:55:32.543619+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:55:32.543619+0000","last_undegraded":"2024-04-17T07:55:32.543619+0000","last_fullsized":"2024-04-17T07:55:32.543619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540350+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:55:32.540350+0000","last_peered":"2024-04-17T07:55:32.540350+0000","last_clean":"2024-04-17T07:55:32.540350+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:55:32.540350+0000","last_undegraded":"2024-04-17T07:55:32.540350+0000","last_fullsized":"2024-04-17T07:55:32.540350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607116+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:55:31.607116+0000","last_peered":"2024-04-17T07:55:31.607116+0000","last_clean":"2024-04-17T07:55:31.607116+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:55:31.607116+0000","last_undegraded":"2024-04-17T07:55:31.607116+0000","last_fullsized":"2024-04-17T07:55:31.607116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604224+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:55:31.604224+0000","last_peered":"2024-04-17T07:55:31.604224+0000","last_clean":"2024-04-17T07:55:31.604224+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:55:31.604224+0000","last_undegraded":"2024-04-17T07:55:31.604224+0000","last_fullsized":"2024-04-17T07:55:31.604224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540248+0000","last_change":"2024-04-17T07:55:26.354251+0000","last_active":"2024-04-17T07:55:32.540248+0000","last_peered":"2024-04-17T07:55:32.540248+0000","last_clean":"2024-04-17T07:55:32.540248+0000","last_became_active":"2024-04-17T07:55:26.353856+0000","last_became_peered":"2024-04-17T07:55:26.353856+0000","last_unstale":"2024-04-17T07:55:32.540248+0000","last_undegraded":"2024-04-17T07:55:32.540248+0000","last_fullsized":"2024-04-17T07:55:32.540248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:48:25.782521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606547+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:55:31.606547+0000","last_peered":"2024-04-17T07:55:31.606547+0000","last_clean":"2024-04-17T07:55:31.606547+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:55:31.606547+0000","last_undegraded":"2024-04-17T07:55:31.606547+0000","last_fullsized":"2024-04-17T07:55:31.606547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539010+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:55:32.539010+0000","last_peered":"2024-04-17T07:55:32.539010+0000","last_clean":"2024-04-17T07:55:32.539010+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:55:32.539010+0000","last_undegraded":"2024-04-17T07:55:32.539010+0000","last_fullsized":"2024-04-17T07:55:32.539010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:54.534187+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:55:54.534187+0000","last_peered":"2024-04-17T07:55:54.534187+0000","last_clean":"2024-04-17T07:55:54.534187+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:55:54.534187+0000","last_undegraded":"2024-04-17T07:55:54.534187+0000","last_fullsized":"2024-04-17T07:55:54.534187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604782+0000","last_change":"2024-04-17T07:55:30.606795+0000","last_active":"2024-04-17T07:55:31.604782+0000","last_peered":"2024-04-17T07:55:31.604782+0000","last_clean":"2024-04-17T07:55:31.604782+0000","last_became_active":"2024-04-17T07:55:30.606351+0000","last_became_peered":"2024-04-17T07:55:30.606351+0000","last_unstale":"2024-04-17T07:55:31.604782+0000","last_undegraded":"2024-04-17T07:55:31.604782+0000","last_fullsized":"2024-04-17T07:55:31.604782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:23.481285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779560+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:55:31.779560+0000","last_peered":"2024-04-17T07:55:31.779560+0000","last_clean":"2024-04-17T07:55:31.779560+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:55:31.779560+0000","last_undegraded":"2024-04-17T07:55:31.779560+0000","last_fullsized":"2024-04-17T07:55:31.779560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:54.530224+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:55:54.530224+0000","last_peered":"2024-04-17T07:55:54.530224+0000","last_clean":"2024-04-17T07:55:54.530224+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:55:54.530224+0000","last_undegraded":"2024-04-17T07:55:54.530224+0000","last_fullsized":"2024-04-17T07:55:54.530224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":347,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.603967+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:55:31.603967+0000","last_peered":"2024-04-17T07:55:31.603967+0000","last_clean":"2024-04-17T07:55:31.603967+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:55:31.603967+0000","last_undegraded":"2024-04-17T07:55:31.603967+0000","last_fullsized":"2024-04-17T07:55:31.603967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539640+0000","last_change":"2024-04-17T07:55:30.600998+0000","last_active":"2024-04-17T07:55:32.539640+0000","last_peered":"2024-04-17T07:55:32.539640+0000","last_clean":"2024-04-17T07:55:32.539640+0000","last_became_active":"2024-04-17T07:55:30.600338+0000","last_became_peered":"2024-04-17T07:55:30.600338+0000","last_unstale":"2024-04-17T07:55:32.539640+0000","last_undegraded":"2024-04-17T07:55:32.539640+0000","last_fullsized":"2024-04-17T07:55:32.539640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:47.103983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606170+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:55:31.606170+0000","last_peered":"2024-04-17T07:55:31.606170+0000","last_clean":"2024-04-17T07:55:31.606170+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:55:31.606170+0000","last_undegraded":"2024-04-17T07:55:31.606170+0000","last_fullsized":"2024-04-17T07:55:31.606170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604042+0000","last_change":"2024-04-17T07:55:25.507838+0000","last_active":"2024-04-17T07:55:31.604042+0000","last_peered":"2024-04-17T07:55:31.604042+0000","last_clean":"2024-04-17T07:55:31.604042+0000","last_became_active":"2024-04-17T07:55:25.507101+0000","last_became_peered":"2024-04-17T07:55:25.507101+0000","last_unstale":"2024-04-17T07:55:31.604042+0000","last_undegraded":"2024-04-17T07:55:31.604042+0000","last_fullsized":"2024-04-17T07:55:31.604042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:52:24.649804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539881+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:55:32.539881+0000","last_peered":"2024-04-17T07:55:32.539881+0000","last_clean":"2024-04-17T07:55:32.539881+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:55:32.539881+0000","last_undegraded":"2024-04-17T07:55:32.539881+0000","last_fullsized":"2024-04-17T07:55:32.539881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541626+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:55:32.541626+0000","last_peered":"2024-04-17T07:55:32.541626+0000","last_clean":"2024-04-17T07:55:32.541626+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:55:32.541626+0000","last_undegraded":"2024-04-17T07:55:32.541626+0000","last_fullsized":"2024-04-17T07:55:32.541626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607478+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:55:31.607478+0000","last_peered":"2024-04-17T07:55:31.607478+0000","last_clean":"2024-04-17T07:55:31.607478+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:55:31.607478+0000","last_undegraded":"2024-04-17T07:55:31.607478+0000","last_fullsized":"2024-04-17T07:55:31.607478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779431+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:55:31.779431+0000","last_peered":"2024-04-17T07:55:31.779431+0000","last_clean":"2024-04-17T07:55:31.779431+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:55:31.779431+0000","last_undegraded":"2024-04-17T07:55:31.779431+0000","last_fullsized":"2024-04-17T07:55:31.779431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.734038+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:55:53.734038+0000","last_peered":"2024-04-17T07:55:53.734038+0000","last_clean":"2024-04-17T07:55:53.734038+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:55:53.734038+0000","last_undegraded":"2024-04-17T07:55:53.734038+0000","last_fullsized":"2024-04-17T07:55:53.734038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780350+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:55:31.780350+0000","last_peered":"2024-04-17T07:55:31.780350+0000","last_clean":"2024-04-17T07:55:31.780350+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:55:31.780350+0000","last_undegraded":"2024-04-17T07:55:31.780350+0000","last_fullsized":"2024-04-17T07:55:31.780350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608676+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:55:31.608676+0000","last_peered":"2024-04-17T07:55:31.608676+0000","last_clean":"2024-04-17T07:55:31.608676+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:55:31.608676+0000","last_undegraded":"2024-04-17T07:55:31.608676+0000","last_fullsized":"2024-04-17T07:55:31.608676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.720331+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:55:52.720331+0000","last_peered":"2024-04-17T07:55:52.720331+0000","last_clean":"2024-04-17T07:55:52.720331+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:55:52.720331+0000","last_undegraded":"2024-04-17T07:55:52.720331+0000","last_fullsized":"2024-04-17T07:55:52.720331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541418+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:55:32.541418+0000","last_peered":"2024-04-17T07:55:32.541418+0000","last_clean":"2024-04-17T07:55:32.541418+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:55:32.541418+0000","last_undegraded":"2024-04-17T07:55:32.541418+0000","last_fullsized":"2024-04-17T07:55:32.541418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541337+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:55:32.541337+0000","last_peered":"2024-04-17T07:55:32.541337+0000","last_clean":"2024-04-17T07:55:32.541337+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:55:32.541337+0000","last_undegraded":"2024-04-17T07:55:32.541337+0000","last_fullsized":"2024-04-17T07:55:32.541337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.723256+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:55:52.723256+0000","last_peered":"2024-04-17T07:55:52.723256+0000","last_clean":"2024-04-17T07:55:52.723256+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:55:52.723256+0000","last_undegraded":"2024-04-17T07:55:52.723256+0000","last_fullsized":"2024-04-17T07:55:52.723256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540686+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:55:32.540686+0000","last_peered":"2024-04-17T07:55:32.540686+0000","last_clean":"2024-04-17T07:55:32.540686+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:55:32.540686+0000","last_undegraded":"2024-04-17T07:55:32.540686+0000","last_fullsized":"2024-04-17T07:55:32.540686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.544257+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:55:32.544257+0000","last_peered":"2024-04-17T07:55:32.544257+0000","last_clean":"2024-04-17T07:55:32.544257+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:55:32.544257+0000","last_undegraded":"2024-04-17T07:55:32.544257+0000","last_fullsized":"2024-04-17T07:55:32.544257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.724264+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:55:52.724264+0000","last_peered":"2024-04-17T07:55:52.724264+0000","last_clean":"2024-04-17T07:55:52.724264+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:55:52.724264+0000","last_undegraded":"2024-04-17T07:55:52.724264+0000","last_fullsized":"2024-04-17T07:55:52.724264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539715+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:55:32.539715+0000","last_peered":"2024-04-17T07:55:32.539715+0000","last_clean":"2024-04-17T07:55:32.539715+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:55:32.539715+0000","last_undegraded":"2024-04-17T07:55:32.539715+0000","last_fullsized":"2024-04-17T07:55:32.539715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541192+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:55:32.541192+0000","last_peered":"2024-04-17T07:55:32.541192+0000","last_clean":"2024-04-17T07:55:32.541192+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:55:32.541192+0000","last_undegraded":"2024-04-17T07:55:32.541192+0000","last_fullsized":"2024-04-17T07:55:32.541192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608630+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:55:31.608630+0000","last_peered":"2024-04-17T07:55:31.608630+0000","last_clean":"2024-04-17T07:55:31.608630+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:55:31.608630+0000","last_undegraded":"2024-04-17T07:55:31.608630+0000","last_fullsized":"2024-04-17T07:55:31.608630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780196+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:55:31.780196+0000","last_peered":"2024-04-17T07:55:31.780196+0000","last_clean":"2024-04-17T07:55:31.780196+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:55:31.780196+0000","last_undegraded":"2024-04-17T07:55:31.780196+0000","last_fullsized":"2024-04-17T07:55:31.780196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.541344+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:55:32.541344+0000","last_peered":"2024-04-17T07:55:32.541344+0000","last_clean":"2024-04-17T07:55:32.541344+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:55:32.541344+0000","last_undegraded":"2024-04-17T07:55:32.541344+0000","last_fullsized":"2024-04-17T07:55:32.541344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:52.720000+0000","last_change":"2024-04-17T07:55:25.507578+0000","last_active":"2024-04-17T07:55:52.720000+0000","last_peered":"2024-04-17T07:55:52.720000+0000","last_clean":"2024-04-17T07:55:52.720000+0000","last_became_active":"2024-04-17T07:55:25.507219+0000","last_became_peered":"2024-04-17T07:55:25.507219+0000","last_unstale":"2024-04-17T07:55:52.720000+0000","last_undegraded":"2024-04-17T07:55:52.720000+0000","last_fullsized":"2024-04-17T07:55:52.720000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:24.445538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604356+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:55:31.604356+0000","last_peered":"2024-04-17T07:55:31.604356+0000","last_clean":"2024-04-17T07:55:31.604356+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:55:31.604356+0000","last_undegraded":"2024-04-17T07:55:31.604356+0000","last_fullsized":"2024-04-17T07:55:31.604356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.539899+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:55:32.539899+0000","last_peered":"2024-04-17T07:55:32.539899+0000","last_clean":"2024-04-17T07:55:32.539899+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:55:32.539899+0000","last_undegraded":"2024-04-17T07:55:32.539899+0000","last_fullsized":"2024-04-17T07:55:32.539899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:53.736779+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:55:53.736779+0000","last_peered":"2024-04-17T07:55:53.736779+0000","last_clean":"2024-04-17T07:55:53.736779+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:55:53.736779+0000","last_undegraded":"2024-04-17T07:55:53.736779+0000","last_fullsized":"2024-04-17T07:55:53.736779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604849+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:55:31.604849+0000","last_peered":"2024-04-17T07:55:31.604849+0000","last_clean":"2024-04-17T07:55:31.604849+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:55:31.604849+0000","last_undegraded":"2024-04-17T07:55:31.604849+0000","last_fullsized":"2024-04-17T07:55:31.604849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540170+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:55:32.540170+0000","last_peered":"2024-04-17T07:55:32.540170+0000","last_clean":"2024-04-17T07:55:32.540170+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:55:32.540170+0000","last_undegraded":"2024-04-17T07:55:32.540170+0000","last_fullsized":"2024-04-17T07:55:32.540170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540423+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:55:32.540423+0000","last_peered":"2024-04-17T07:55:32.540423+0000","last_clean":"2024-04-17T07:55:32.540423+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:55:32.540423+0000","last_undegraded":"2024-04-17T07:55:32.540423+0000","last_fullsized":"2024-04-17T07:55:32.540423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604544+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:55:31.604544+0000","last_peered":"2024-04-17T07:55:31.604544+0000","last_clean":"2024-04-17T07:55:31.604544+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:55:31.604544+0000","last_undegraded":"2024-04-17T07:55:31.604544+0000","last_fullsized":"2024-04-17T07:55:31.604544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.538743+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:55:32.538743+0000","last_peered":"2024-04-17T07:55:32.538743+0000","last_clean":"2024-04-17T07:55:32.538743+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:55:32.538743+0000","last_undegraded":"2024-04-17T07:55:32.538743+0000","last_fullsized":"2024-04-17T07:55:32.538743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607279+0000","last_change":"2024-04-17T07:55:25.530309+0000","last_active":"2024-04-17T07:55:31.607279+0000","last_peered":"2024-04-17T07:55:31.607279+0000","last_clean":"2024-04-17T07:55:31.607279+0000","last_became_active":"2024-04-17T07:55:25.529880+0000","last_became_peered":"2024-04-17T07:55:25.529880+0000","last_unstale":"2024-04-17T07:55:31.607279+0000","last_undegraded":"2024-04-17T07:55:31.607279+0000","last_fullsized":"2024-04-17T07:55:31.607279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:00.265792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.604799+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:55:31.604799+0000","last_peered":"2024-04-17T07:55:31.604799+0000","last_clean":"2024-04-17T07:55:31.604799+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:55:31.604799+0000","last_undegraded":"2024-04-17T07:55:31.604799+0000","last_fullsized":"2024-04-17T07:55:31.604799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540341+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:55:32.540341+0000","last_peered":"2024-04-17T07:55:32.540341+0000","last_clean":"2024-04-17T07:55:32.540341+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:55:32.540341+0000","last_undegraded":"2024-04-17T07:55:32.540341+0000","last_fullsized":"2024-04-17T07:55:32.540341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608749+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:55:31.608749+0000","last_peered":"2024-04-17T07:55:31.608749+0000","last_clean":"2024-04-17T07:55:31.608749+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:55:31.608749+0000","last_undegraded":"2024-04-17T07:55:31.608749+0000","last_fullsized":"2024-04-17T07:55:31.608749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.608521+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:55:31.608521+0000","last_peered":"2024-04-17T07:55:31.608521+0000","last_clean":"2024-04-17T07:55:31.608521+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:55:31.608521+0000","last_undegraded":"2024-04-17T07:55:31.608521+0000","last_fullsized":"2024-04-17T07:55:31.608521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540969+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:55:32.540969+0000","last_peered":"2024-04-17T07:55:32.540969+0000","last_clean":"2024-04-17T07:55:32.540969+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:55:32.540969+0000","last_undegraded":"2024-04-17T07:55:32.540969+0000","last_fullsized":"2024-04-17T07:55:32.540969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.607386+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:55:31.607386+0000","last_peered":"2024-04-17T07:55:31.607386+0000","last_clean":"2024-04-17T07:55:31.607386+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:55:31.607386+0000","last_undegraded":"2024-04-17T07:55:31.607386+0000","last_fullsized":"2024-04-17T07:55:31.607386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540151+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:55:32.540151+0000","last_peered":"2024-04-17T07:55:32.540151+0000","last_clean":"2024-04-17T07:55:32.540151+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:55:32.540151+0000","last_undegraded":"2024-04-17T07:55:32.540151+0000","last_fullsized":"2024-04-17T07:55:32.540151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.609264+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:55:31.609264+0000","last_peered":"2024-04-17T07:55:31.609264+0000","last_clean":"2024-04-17T07:55:31.609264+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:55:31.609264+0000","last_undegraded":"2024-04-17T07:55:31.609264+0000","last_fullsized":"2024-04-17T07:55:31.609264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.780721+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:55:31.780721+0000","last_peered":"2024-04-17T07:55:31.780721+0000","last_clean":"2024-04-17T07:55:31.780721+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:55:31.780721+0000","last_undegraded":"2024-04-17T07:55:31.780721+0000","last_fullsized":"2024-04-17T07:55:31.780721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.538936+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:55:32.538936+0000","last_peered":"2024-04-17T07:55:32.538936+0000","last_clean":"2024-04-17T07:55:32.538936+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:55:32.538936+0000","last_undegraded":"2024-04-17T07:55:32.538936+0000","last_fullsized":"2024-04-17T07:55:32.538936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.779608+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:55:31.779608+0000","last_peered":"2024-04-17T07:55:31.779608+0000","last_clean":"2024-04-17T07:55:31.779608+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:55:31.779608+0000","last_undegraded":"2024-04-17T07:55:31.779608+0000","last_fullsized":"2024-04-17T07:55:31.779608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540661+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:55:32.540661+0000","last_peered":"2024-04-17T07:55:32.540661+0000","last_clean":"2024-04-17T07:55:32.540661+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:55:32.540661+0000","last_undegraded":"2024-04-17T07:55:32.540661+0000","last_fullsized":"2024-04-17T07:55:32.540661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:31.606729+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:55:31.606729+0000","last_peered":"2024-04-17T07:55:31.606729+0000","last_clean":"2024-04-17T07:55:31.606729+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:55:31.606729+0000","last_undegraded":"2024-04-17T07:55:31.606729+0000","last_fullsized":"2024-04-17T07:55:31.606729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-17T07:55:32.540136+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:55:32.540136+0000","last_peered":"2024-04-17T07:55:32.540136+0000","last_clean":"2024-04-17T07:55:32.540136+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:55:32.540136+0000","last_undegraded":"2024-04-17T07:55:32.540136+0000","last_fullsized":"2024-04-17T07:55:32.540136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46298,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":34,"num_read_kb":48,"num_write":135,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":118047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383701,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":481893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645337,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1092693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808862,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070500,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":477841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":487657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":478693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":520862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":477319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1609728,"data_stored":1073546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":43647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":43875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:55:59.151 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T07:55:59.152 INFO:tasks.cephadm:Setting up client nodes... 2024-04-17T07:55:59.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-17T07:55:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:58 smithi053 ceph-mon[36437]: pgmap v337: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-17T07:55:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:58 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3018536782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:55:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:00.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:59 smithi046 ceph-mon[32038]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:56:00.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:55:59 smithi046 ceph-mon[32038]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:56:00.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:59 smithi062 ceph-mon[36507]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:56:00.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:55:59 smithi062 ceph-mon[36507]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:56:00.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:59 smithi053 ceph-mon[36437]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:56:00.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:55:59 smithi053 ceph-mon[36437]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:56:01.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:00 smithi062 ceph-mon[36507]: pgmap v338: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-17T07:56:01.086 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:00 smithi046 ceph-mon[32038]: pgmap v338: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-17T07:56:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:00 smithi053 ceph-mon[36437]: pgmap v338: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 -- 172.21.15.46:0/2932188884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 msgr2=0x7f8fbc105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2932188884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 0x7f8fbc105120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00099c0 tx=0x7f8fb0009cd0 comp rx=0 tx=0).stop 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 -- 172.21.15.46:0/2932188884 shutdown_connections 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2932188884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fbc1068a0 0x7f8fbc10d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2932188884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 0x7f8fbc105120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2932188884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fbc105f00 0x7f8fbc106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 -- 172.21.15.46:0/2932188884 >> 172.21.15.46:0/2932188884 conn(0x7f8fbc1002a0 msgr2=0x7f8fbc1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 -- 172.21.15.46:0/2932188884 shutdown_connections 2024-04-17T07:56:01.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.949+0000 7f8fc3600700 1 -- 172.21.15.46:0/2932188884 wait complete. 2024-04-17T07:56:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 Processor -- start 2024-04-17T07:56:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 -- start start 2024-04-17T07:56:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 0x7f8fbc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fbc105f00 0x7f8fbc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fbc1068a0 0x7f8fbc1a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc110910 con 0x7f8fbc104d00 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc110790 con 0x7f8fbc1068a0 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc3600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc110a90 con 0x7f8fbc105f00 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.951+0000 7f8fc0c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 0x7f8fbc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fbbe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fbc105f00 0x7f8fbc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc1600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fbc1068a0 0x7f8fbc1a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc0c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 0x7f8fbc19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35894/0 (socket says 172.21.15.46:35894) 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc0c00700 1 -- 172.21.15.46:0/2929706818 learned_addr learned my addr 172.21.15.46:0/2929706818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc1600700 1 -- 172.21.15.46:0/2929706818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fbc105f00 msgr2=0x7f8fbc19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc1600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fbc105f00 0x7f8fbc19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc1600700 1 -- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 msgr2=0x7f8fbc19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:01.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc1600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 0x7f8fbc19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc1600700 1 -- 172.21.15.46:0/2929706818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc1a2a90 con 0x7f8fbc1068a0 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.952+0000 7f8fc1600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fbc1068a0 0x7f8fbc1a23f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400efb0 tx=0x7f8fb400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.953+0000 7f8fb9600700 1 -- 172.21.15.46:0/2929706818 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003a00 con 0x7f8fbc1068a0 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.953+0000 7f8fb9600700 1 -- 172.21.15.46:0/2929706818 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb4003ba0 con 0x7f8fbc1068a0 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.953+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc1a2d80 con 0x7f8fbc1068a0 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.953+0000 7f8fb9600700 1 -- 172.21.15.46:0/2929706818 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4004840 con 0x7f8fbc1068a0 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.953+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc1a3240 con 0x7f8fbc1068a0 2024-04-17T07:56:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.954+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc050f90 con 0x7f8fbc1068a0 2024-04-17T07:56:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.956+0000 7f8fb9600700 1 -- 172.21.15.46:0/2929706818 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8fb4012050 con 0x7f8fbc1068a0 2024-04-17T07:56:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.957+0000 7f8fb9600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8fa40a3e00 0x7f8fa40a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.957+0000 7f8fb9600700 1 -- 172.21.15.46:0/2929706818 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8fb40cfff0 con 0x7f8fbc1068a0 2024-04-17T07:56:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.957+0000 7f8fc0c00700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8fa40a3e00 0x7f8fa40a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.958+0000 7f8fc0c00700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8fa40a3e00 0x7f8fa40a62c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00038c0 tx=0x7f8fb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:01.962+0000 7f8fb9600700 1 -- 172.21.15.46:0/2929706818 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8fb4022060 con 0x7f8fbc1068a0 2024-04-17T07:56:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.288+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8fbc063660 con 0x7f8fbc1068a0 2024-04-17T07:56:02.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.296+0000 7f8fb9600700 1 -- 172.21.15.46:0/2929706818 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f8fb4010040 con 0x7f8fbc1068a0 2024-04-17T07:56:02.297 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-17T07:56:02.297 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSgB9maXphERAArjwyThU01qUkn/h91mzoyQ== 2024-04-17T07:56:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8fa40a3e00 msgr2=0x7f8fa40a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8fa40a3e00 0x7f8fa40a62c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00038c0 tx=0x7f8fb0005fb0 comp rx=0 tx=0).stop 2024-04-17T07:56:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fbc1068a0 msgr2=0x7f8fbc1a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fbc1068a0 0x7f8fbc1a23f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400efb0 tx=0x7f8fb400c3a0 comp rx=0 tx=0).stop 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 shutdown_connections 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8fa40a3e00 0x7f8fa40a62c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fbc1068a0 0x7f8fbc1a23f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fbc104d00 0x7f8fbc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 --2- 172.21.15.46:0/2929706818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fbc105f00 0x7f8fbc19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 >> 172.21.15.46:0/2929706818 conn(0x7f8fbc1002a0 msgr2=0x7f8fbc109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 shutdown_connections 2024-04-17T07:56:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:02.300+0000 7f8fc3600700 1 -- 172.21.15.46:0/2929706818 wait complete. 2024-04-17T07:56:02.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:02 smithi046 ceph-mon[32038]: pgmap v339: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-04-17T07:56:02.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:02 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2929706818' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:56:02.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:02 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:56:02.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:02 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T07:56:03.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:02 smithi062 ceph-mon[36507]: pgmap v339: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-04-17T07:56:03.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:02 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2929706818' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:56:03.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:02 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:56:03.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:02 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T07:56:03.080 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:56:03.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-17T07:56:03.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:02 smithi053 ceph-mon[36437]: pgmap v339: 129 pgs: 129 active+clean; 622 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-04-17T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:02 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2929706818' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:02 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:56:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:02 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T07:56:03.250 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T07:56:03.250 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-17T07:56:03.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph mgr dump --format=json 2024-04-17T07:56:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:05.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:04 smithi062 ceph-mon[36507]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-17T07:56:05.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:04 smithi046 ceph-mon[32038]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-17T07:56:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:04 smithi053 ceph-mon[36437]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-17T07:56:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 -- 172.21.15.46:0/2867110697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66600778a0 msgr2=0x7f66601133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 --2- 172.21.15.46:0/2867110697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66600778a0 0x7f66601133c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f66480098f0 tx=0x7f6648009c00 comp rx=0 tx=0).stop 2024-04-17T07:56:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 -- 172.21.15.46:0/2867110697 shutdown_connections 2024-04-17T07:56:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 --2- 172.21.15.46:0/2867110697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6660113900 0x7f6660115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 --2- 172.21.15.46:0/2867110697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6660076f80 0x7f6660077360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 --2- 172.21.15.46:0/2867110697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66600778a0 0x7f66601133c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 -- 172.21.15.46:0/2867110697 >> 172.21.15.46:0/2867110697 conn(0x7f6660100730 msgr2=0x7f6660102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 -- 172.21.15.46:0/2867110697 shutdown_connections 2024-04-17T07:56:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.067+0000 7f6665e00700 1 -- 172.21.15.46:0/2867110697 wait complete. 2024-04-17T07:56:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.068+0000 7f6665e00700 1 Processor -- start 2024-04-17T07:56:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6665e00700 1 -- start start 2024-04-17T07:56:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6665e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6660076f80 0x7f66601a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6665e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66600778a0 0x7f66601a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6665e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6660113900 0x7f666019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6665e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66601193d0 con 0x7f66600778a0 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6665e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6660119250 con 0x7f6660076f80 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6665e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6660119550 con 0x7f6660113900 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.069+0000 7f6656000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66600778a0 0x7f66601a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66600778a0 0x7f66601a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35912/0 (socket says 172.21.15.46:35912) 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 -- 172.21.15.46:0/4097930135 learned_addr learned my addr 172.21.15.46:0/4097930135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f665ea00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6660076f80 0x7f66601a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f665f400700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6660113900 0x7f666019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 -- 172.21.15.46:0/4097930135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6660113900 msgr2=0x7f666019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6660113900 0x7f666019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 -- 172.21.15.46:0/4097930135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6660076f80 msgr2=0x7f66601a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6660076f80 0x7f66601a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 -- 172.21.15.46:0/4097930135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666019ffd0 con 0x7f66600778a0 2024-04-17T07:56:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f665ea00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6660076f80 0x7f66601a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.070+0000 7f6656000700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66600778a0 0x7f66601a5b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6648009230 tx=0x7f664800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.071+0000 7f6664a00700 1 -- 172.21.15.46:0/4097930135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648005180 con 0x7f66600778a0 2024-04-17T07:56:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.071+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66601a0260 con 0x7f66600778a0 2024-04-17T07:56:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.071+0000 7f6664a00700 1 -- 172.21.15.46:0/4097930135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f664800b560 con 0x7f66600778a0 2024-04-17T07:56:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.071+0000 7f6664a00700 1 -- 172.21.15.46:0/4097930135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648003a70 con 0x7f66600778a0 2024-04-17T07:56:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.071+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601ac380 con 0x7f66600778a0 2024-04-17T07:56:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.072+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66600786c0 con 0x7f66600778a0 2024-04-17T07:56:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.074+0000 7f6664a00700 1 -- 172.21.15.46:0/4097930135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6648029060 con 0x7f66600778a0 2024-04-17T07:56:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.076+0000 7f6664a00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66440a3d50 0x7f66440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.076+0000 7f6664a00700 1 -- 172.21.15.46:0/4097930135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f66480d0fe0 con 0x7f66600778a0 2024-04-17T07:56:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.076+0000 7f665ea00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66440a3d50 0x7f66440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.077+0000 7f665ea00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66440a3d50 0x7f66440a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6650009750 tx=0x7f6650009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.081+0000 7f6664a00700 1 -- 172.21.15.46:0/4097930135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6648013030 con 0x7f66600778a0 2024-04-17T07:56:06.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.386+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f666002df90 con 0x7f66600778a0 2024-04-17T07:56:06.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.401+0000 7f6664a00700 1 -- 172.21.15.46:0/4097930135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f664809d690 con 0x7f66600778a0 2024-04-17T07:56:06.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:56:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.406+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66440a3d50 msgr2=0x7f66440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.406+0000 7f6665e00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66440a3d50 0x7f66440a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6650009750 tx=0x7f6650009310 comp rx=0 tx=0).stop 2024-04-17T07:56:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66600778a0 msgr2=0x7f66601a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66600778a0 0x7f66601a5b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6648009230 tx=0x7f664800fa70 comp rx=0 tx=0).stop 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 shutdown_connections 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66440a3d50 0x7f66440a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6660113900 0x7f666019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6660076f80 0x7f66601a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 --2- 172.21.15.46:0/4097930135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66600778a0 0x7f66601a5b10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 >> 172.21.15.46:0/4097930135 conn(0x7f6660100730 msgr2=0x7f6660102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 shutdown_connections 2024-04-17T07:56:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:06.407+0000 7f6665e00700 1 -- 172.21.15.46:0/4097930135 wait complete. 2024-04-17T07:56:07.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:06 smithi062 ceph-mon[36507]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-04-17T07:56:07.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:06 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4097930135' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T07:56:07.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:06 smithi046 ceph-mon[32038]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-04-17T07:56:07.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:06 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4097930135' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T07:56:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:06 smithi053 ceph-mon[36437]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-04-17T07:56:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:06 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4097930135' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T07:56:07.247 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":4172081855},{"type":"v1","addr":"172.21.15.46:6801","nonce":4172081855}]},"active_addr":"172.21.15.46:6801/4172081855","active_change":"2024-04-17T07:47:21.351701+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":177422763}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2597234954}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3467161945}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2345998964}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2083273673}]}]} 2024-04-17T07:56:07.252 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-17T07:56:07.252 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-17T07:56:07.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T07:56:07.757 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:08.947 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:08 smithi046 ceph-mon[32038]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-17T07:56:09.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:08 smithi062 ceph-mon[36507]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-17T07:56:09.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:08 smithi053 ceph-mon[36437]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-17T07:56:09.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 -- 172.21.15.46:0/2468931177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 msgr2=0x7f36d41051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2468931177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d41051c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f36c800b0d0 tx=0x7f36c800b3e0 comp rx=0 tx=0).stop 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 -- 172.21.15.46:0/2468931177 shutdown_connections 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2468931177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4109ec0 0x7f36d4111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2468931177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d41051c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2468931177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4105790 0x7f36d41097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 -- 172.21.15.46:0/2468931177 >> 172.21.15.46:0/2468931177 conn(0x7f36d4100670 msgr2=0x7f36d4102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 -- 172.21.15.46:0/2468931177 shutdown_connections 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.880+0000 7f36d9e00700 1 -- 172.21.15.46:0/2468931177 wait complete. 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.881+0000 7f36d9e00700 1 Processor -- start 2024-04-17T07:56:09.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.881+0000 7f36d9e00700 1 -- start start 2024-04-17T07:56:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d9e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4105790 0x7f36d419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d9e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4109ec0 0x7f36d419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d9e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4114f00 con 0x7f36d4104de0 2024-04-17T07:56:09.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d9e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4114d80 con 0x7f36d4109ec0 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d9e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4115080 con 0x7f36d4105790 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d9400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4109ec0 0x7f36d419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d8a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46790/0 (socket says 172.21.15.46:46790) 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d8a00700 1 -- 172.21.15.46:0/2584266824 learned_addr learned my addr 172.21.15.46:0/2584266824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.882+0000 7f36d3e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4105790 0x7f36d419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d9400700 1 -- 172.21.15.46:0/2584266824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4105790 msgr2=0x7f36d419b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d9400700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4105790 0x7f36d419b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d9400700 1 -- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 msgr2=0x7f36d419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d9400700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d9400700 1 -- 172.21.15.46:0/2584266824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d41a0350 con 0x7f36d4109ec0 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d3e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4105790 0x7f36d419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d8a00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:09.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.883+0000 7f36d9400700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4109ec0 0x7f36d419fc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00e900 tx=0x7f36cc00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.884+0000 7f36d1600700 1 -- 172.21.15.46:0/2584266824 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004220 con 0x7f36d4109ec0 2024-04-17T07:56:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.884+0000 7f36d1600700 1 -- 172.21.15.46:0/2584266824 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc0043c0 con 0x7f36d4109ec0 2024-04-17T07:56:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.884+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d41a0640 con 0x7f36d4109ec0 2024-04-17T07:56:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.884+0000 7f36d1600700 1 -- 172.21.15.46:0/2584266824 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc010430 con 0x7f36d4109ec0 2024-04-17T07:56:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.884+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a7ec0 con 0x7f36d4109ec0 2024-04-17T07:56:09.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.885+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d4069740 con 0x7f36d4109ec0 2024-04-17T07:56:09.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.887+0000 7f36d1600700 1 -- 172.21.15.46:0/2584266824 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f36cc00c780 con 0x7f36d4109ec0 2024-04-17T07:56:09.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.889+0000 7f36d1600700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36bc0a3cd0 0x7f36bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:09.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.889+0000 7f36d1600700 1 -- 172.21.15.46:0/2584266824 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f36cc0d08b0 con 0x7f36d4109ec0 2024-04-17T07:56:09.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.889+0000 7f36d8a00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36bc0a3cd0 0x7f36bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:09.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.890+0000 7f36d8a00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36bc0a3cd0 0x7f36bc0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009fd0 tx=0x7f36c800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:09.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:09.893+0000 7f36d1600700 1 -- 172.21.15.46:0/2584266824 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f36cc09ce90 con 0x7f36d4109ec0 2024-04-17T07:56:10.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.130+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36d402df90 con 0x7f36d4109ec0 2024-04-17T07:56:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.132+0000 7f36d1600700 1 -- 172.21.15.46:0/2584266824 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f36cc09c790 con 0x7f36d4109ec0 2024-04-17T07:56:10.133 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:56:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36bc0a3cd0 msgr2=0x7f36bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36bc0a3cd0 0x7f36bc0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009fd0 tx=0x7f36c800b040 comp rx=0 tx=0).stop 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4109ec0 msgr2=0x7f36d419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4109ec0 0x7f36d419fc10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00e900 tx=0x7f36cc00ec10 comp rx=0 tx=0).stop 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 shutdown_connections 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f36bc0a3cd0 0x7f36bc0a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36d4109ec0 0x7f36d419fc10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36d4104de0 0x7f36d419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 --2- 172.21.15.46:0/2584266824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36d4105790 0x7f36d419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.135+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 >> 172.21.15.46:0/2584266824 conn(0x7f36d4100670 msgr2=0x7f36d4102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.136+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 shutdown_connections 2024-04-17T07:56:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:10.136+0000 7f36d9e00700 1 -- 172.21.15.46:0/2584266824 wait complete. 2024-04-17T07:56:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:10 smithi062 ceph-mon[36507]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-17T07:56:11.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:10 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2584266824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:56:11.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:10 smithi046 ceph-mon[32038]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-17T07:56:11.060 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:10 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2584266824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:56:11.134 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-17T07:56:11.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:10 smithi053 ceph-mon[36437]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-17T07:56:11.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:10 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2584266824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:56:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:12.975 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:12 smithi046 ceph-mon[32038]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-17T07:56:13.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:12 smithi062 ceph-mon[36507]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-17T07:56:13.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:12 smithi053 ceph-mon[36437]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-17T07:56:13.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.821+0000 7f8399200700 1 -- 172.21.15.46:0/3863234707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839406b080 msgr2=0x7f839410efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:13.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.821+0000 7f8399200700 1 --2- 172.21.15.46:0/3863234707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839406b080 0x7f839410efc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8380009990 tx=0x7f8380009ca0 comp rx=0 tx=0).stop 2024-04-17T07:56:13.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.821+0000 7f8399200700 1 -- 172.21.15.46:0/3863234707 shutdown_connections 2024-04-17T07:56:13.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.821+0000 7f8399200700 1 --2- 172.21.15.46:0/3863234707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839410f500 0x7f83941118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:13.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.821+0000 7f8399200700 1 --2- 172.21.15.46:0/3863234707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f839406a760 0x7f839406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:13.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.821+0000 7f8399200700 1 --2- 172.21.15.46:0/3863234707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839406b080 0x7f839410efc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.821+0000 7f8399200700 1 -- 172.21.15.46:0/3863234707 >> 172.21.15.46:0/3863234707 conn(0x7f8394078900 msgr2=0x7f8394078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.822+0000 7f8399200700 1 -- 172.21.15.46:0/3863234707 shutdown_connections 2024-04-17T07:56:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.822+0000 7f8399200700 1 -- 172.21.15.46:0/3863234707 wait complete. 2024-04-17T07:56:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 Processor -- start 2024-04-17T07:56:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 -- start start 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839406a760 0x7f839419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f839406b080 0x7f839419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839410f500 0x7f839419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8394114fd0 con 0x7f839406a760 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8394114e50 con 0x7f839406b080 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8399200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8394115150 con 0x7f839410f500 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.823+0000 7f8391e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839406a760 0x7f839419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8391e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839406a760 0x7f839419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46808/0 (socket says 172.21.15.46:46808) 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8391e00700 1 -- 172.21.15.46:0/3023838722 learned_addr learned my addr 172.21.15.46:0/3023838722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8392800700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839410f500 0x7f839419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8391400700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f839406b080 0x7f839419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8392800700 1 -- 172.21.15.46:0/3023838722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f839406b080 msgr2=0x7f839419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8392800700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f839406b080 0x7f839419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8392800700 1 -- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839406a760 msgr2=0x7f839419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8392800700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839406a760 0x7f839419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8392800700 1 -- 172.21.15.46:0/3023838722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83941a0460 con 0x7f839410f500 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.824+0000 7f8391e00700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839406a760 0x7f839419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.825+0000 7f8392800700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839410f500 0x7f839419fd20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ef90 tx=0x7f838c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.825+0000 7f838aa00700 1 -- 172.21.15.46:0/3023838722 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c0092e0 con 0x7f839410f500 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.825+0000 7f838aa00700 1 -- 172.21.15.46:0/3023838722 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c004bb0 con 0x7f839410f500 2024-04-17T07:56:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.825+0000 7f838aa00700 1 -- 172.21.15.46:0/3023838722 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c004280 con 0x7f839410f500 2024-04-17T07:56:13.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.825+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83941a06f0 con 0x7f839410f500 2024-04-17T07:56:13.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.826+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83941a7fd0 con 0x7f839410f500 2024-04-17T07:56:13.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.827+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8394069740 con 0x7f839410f500 2024-04-17T07:56:13.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.828+0000 7f838aa00700 1 -- 172.21.15.46:0/3023838722 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f838c01b030 con 0x7f839410f500 2024-04-17T07:56:13.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.831+0000 7f838aa00700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83780a3d50 0x7f83780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:13.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.831+0000 7f838aa00700 1 -- 172.21.15.46:0/3023838722 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f838c0d0290 con 0x7f839410f500 2024-04-17T07:56:13.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.831+0000 7f8391e00700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83780a3d50 0x7f83780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:13.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.831+0000 7f8391e00700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83780a3d50 0x7f83780a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8384011590 tx=0x7f8384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:13.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:13.836+0000 7f838aa00700 1 -- 172.21.15.46:0/3023838722 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f838c022020 con 0x7f839410f500 2024-04-17T07:56:14.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.067+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f839402df90 con 0x7f839410f500 2024-04-17T07:56:14.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.070+0000 7f838aa00700 1 -- 172.21.15.46:0/3023838722 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23991 (secure 0 0 0) 0x7f838c010040 con 0x7f839410f500 2024-04-17T07:56:14.071 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:56:14.071 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:55:31.064785+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83780a3d50 msgr2=0x7f83780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83780a3d50 0x7f83780a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8384011590 tx=0x7f8384009260 comp rx=0 tx=0).stop 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839410f500 msgr2=0x7f839419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839410f500 0x7f839419fd20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f838c00ef90 tx=0x7f838c00c3a0 comp rx=0 tx=0).stop 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 shutdown_connections 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83780a3d50 0x7f83780a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839410f500 0x7f839419fd20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839406a760 0x7f839419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:14.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 --2- 172.21.15.46:0/3023838722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f839406b080 0x7f839419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:14.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 >> 172.21.15.46:0/3023838722 conn(0x7f8394078900 msgr2=0x7f8394110950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:14.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 shutdown_connections 2024-04-17T07:56:14.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:14.073+0000 7f8399200700 1 -- 172.21.15.46:0/3023838722 wait complete. 2024-04-17T07:56:14.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-17T07:56:14.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-17T07:56:14.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-17T07:56:14.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-17T07:56:14.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-17T07:56:14.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-17T07:56:14.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-17T07:56:14.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-17T07:56:14.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-17T07:56:14.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-17T07:56:14.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-17T07:56:14.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-17T07:56:15.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:14 smithi062 ceph-mon[36507]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-17T07:56:15.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:14 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3023838722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:56:15.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:14 smithi046 ceph-mon[32038]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-17T07:56:15.074 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:14 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3023838722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:56:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:14 smithi053 ceph-mon[36437]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-17T07:56:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:14 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3023838722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:56:16.125 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.438 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.449 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.450 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.456 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.457 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.461 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.469 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.475 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:17.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:16 smithi062 ceph-mon[36507]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-17T07:56:17.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:16 smithi046 ceph-mon[32038]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-17T07:56:17.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:16 smithi053 ceph-mon[36437]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-17T07:56:18.850 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:18 smithi046 ceph-mon[32038]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-17T07:56:19.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:18 smithi062 ceph-mon[36507]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-17T07:56:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:18 smithi053 ceph-mon[36437]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-17T07:56:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:21.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:20 smithi062 ceph-mon[36507]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T07:56:21.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:20 smithi046 ceph-mon[32038]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T07:56:21.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:20 smithi053 ceph-mon[36437]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-17T07:56:22.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-17T07:56:22.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-04-17T07:56:22.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-17T07:56:22.111 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-17T07:56:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-17T07:56:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-17T07:56:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-17T07:56:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-17T07:56:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-17T07:56:22.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-17T07:56:22.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-17T07:56:22.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-04-17T07:56:22.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-04-17T07:56:22.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-17T07:56:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:23.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T07:56:23.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-17T07:56:23.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-17T07:56:23.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-17T07:56:23.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-17T07:56:23.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-17T07:56:23.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-17T07:56:23.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-17T07:56:23.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-04-17T07:56:23.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-17T07:56:23.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-17T07:56:23.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:22 smithi046 ceph-mon[32038]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T07:56:23.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T07:56:23.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-17T07:56:23.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-17T07:56:23.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-17T07:56:23.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:22 smithi053 ceph-mon[36437]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-04-17T07:56:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-17T07:56:23.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:22 smithi062 ceph-mon[36507]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T07:56:24.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:23 smithi046 ceph-mon[32038]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T07:56:24.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:23 smithi053 ceph-mon[36437]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T07:56:24.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:23 smithi062 ceph-mon[36507]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T07:56:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 -- 172.21.15.46:0/3634722612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c06b120 msgr2=0x7f979c06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 --2- 172.21.15.46:0/3634722612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c06b120 0x7f979c06b580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9788009990 tx=0x7f9788009ca0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 -- 172.21.15.46:0/3634722612 shutdown_connections 2024-04-17T07:56:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 --2- 172.21.15.46:0/3634722612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f979c06bac0 0x7f979c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 --2- 172.21.15.46:0/3634722612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f979c108a60 0x7f979c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 --2- 172.21.15.46:0/3634722612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c06b120 0x7f979c06b580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 -- 172.21.15.46:0/3634722612 >> 172.21.15.46:0/3634722612 conn(0x7f979c078900 msgr2=0x7f979c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 -- 172.21.15.46:0/3634722612 shutdown_connections 2024-04-17T07:56:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.814+0000 7f97a1a00700 1 -- 172.21.15.46:0/3634722612 wait complete. 2024-04-17T07:56:24.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.815+0000 7f97a1a00700 1 Processor -- start 2024-04-17T07:56:24.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f97a1a00700 1 -- start start 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f97a1a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c06b120 0x7f979c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f97a1a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f979c06bac0 0x7f979c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f97a1a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f979c108a60 0x7f979c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f97a1a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f979c112d80 con 0x7f979c06bac0 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f97a1a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f979c112c00 con 0x7f979c108a60 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f97a1a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f979c112f00 con 0x7f979c06b120 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f979a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f979c06bac0 0x7f979c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f979a000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f979c06bac0 0x7f979c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56122/0 (socket says 172.21.15.46:56122) 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.816+0000 7f979a000700 1 -- 172.21.15.46:0/660161077 learned_addr learned my addr 172.21.15.46:0/660161077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979aa00700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c06b120 0x7f979c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979b400700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f979c108a60 0x7f979c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979a000700 1 -- 172.21.15.46:0/660161077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c06b120 msgr2=0x7f979c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979a000700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c06b120 0x7f979c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979a000700 1 -- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f979c108a60 msgr2=0x7f979c19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979a000700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f979c108a60 0x7f979c19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979a000700 1 -- 172.21.15.46:0/660161077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c19e280 con 0x7f979c06bac0 2024-04-17T07:56:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979b400700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f979c108a60 0x7f979c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.817+0000 7f979a000700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f979c06bac0 0x7f979c1997b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9788005910 tx=0x7f978800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.818+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788027070 con 0x7f979c06bac0 2024-04-17T07:56:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.818+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c19e510 con 0x7f979c06bac0 2024-04-17T07:56:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.818+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9788004650 con 0x7f979c06bac0 2024-04-17T07:56:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.818+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c1a5df0 con 0x7f979c06bac0 2024-04-17T07:56:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.818+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978800f460 con 0x7f979c06bac0 2024-04-17T07:56:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.819+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f979c0631a0 con 0x7f979c06bac0 2024-04-17T07:56:24.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.822+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f978802a070 con 0x7f979c06bac0 2024-04-17T07:56:24.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.824+0000 7f9793400700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f97800a3d50 0x7f97800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:24.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.824+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f97880d1ea0 con 0x7f979c06bac0 2024-04-17T07:56:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.824+0000 7f979aa00700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f97800a3d50 0x7f97800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.824+0000 7f9793400700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] conn(0x7f97800aeeb0 0x7f97800b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.824+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 --> [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97800b1a20 con 0x7f97800aeeb0 2024-04-17T07:56:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.824+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f97880d2250 con 0x7f979c06bac0 2024-04-17T07:56:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.825+0000 7f979aa00700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f97800a3d50 0x7f97800a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f978c009be0 tx=0x7f978c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.825+0000 7f979b400700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] conn(0x7f97800aeeb0 0x7f97800b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.826+0000 7f979b400700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] conn(0x7f97800aeeb0 0x7f97800b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.826+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== osd.10 v2:172.21.15.53:6824/795360717 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97800b1a20 con 0x7f97800aeeb0 2024-04-17T07:56:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.848+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 --> [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f979c0635d0 con 0x7f97800aeeb0 2024-04-17T07:56:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.849+0000 7f9793400700 1 -- 172.21.15.46:0/660161077 <== osd.10 v2:172.21.15.53:6824/795360717 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f979c0635d0 con 0x7f97800aeeb0 2024-04-17T07:56:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] conn(0x7f97800aeeb0 msgr2=0x7f97800b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] conn(0x7f97800aeeb0 0x7f97800b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f97800a3d50 msgr2=0x7f97800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f97800a3d50 0x7f97800a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f978c009be0 tx=0x7f978c009260 comp rx=0 tx=0).stop 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f979c06bac0 msgr2=0x7f979c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 --2- 172.21.15.46:0/660161077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f979c06bac0 0x7f979c1997b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9788005910 tx=0x7f978800bab0 comp rx=0 tx=0).stop 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f979b400700 1 -- 172.21.15.46:0/660161077 reap_dead start 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 shutdown_connections 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.850+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 >> 172.21.15.46:0/660161077 conn(0x7f979c078900 msgr2=0x7f979c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.851+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 shutdown_connections 2024-04-17T07:56:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:24.851+0000 7f97a1a00700 1 -- 172.21.15.46:0/660161077 wait complete. 2024-04-17T07:56:25.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:24 smithi046 ceph-mon[32038]: pgmap v352: 129 pgs: 4 peering, 125 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s 2024-04-17T07:56:25.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:24 smithi053 ceph-mon[36437]: pgmap v352: 129 pgs: 4 peering, 125 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s 2024-04-17T07:56:25.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:24 smithi062 ceph-mon[36507]: pgmap v352: 129 pgs: 4 peering, 125 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s 2024-04-17T07:56:27.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:26 smithi046 ceph-mon[32038]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:56:27.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:56:27.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:26 smithi053 ceph-mon[36437]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:56:27.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:56:27.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:26 smithi062 ceph-mon[36507]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:56:27.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:56:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:28 smithi046 ceph-mon[32038]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:56:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:28 smithi053 ceph-mon[36437]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:56:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:28 smithi062 ceph-mon[36507]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:56:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:56:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.439+0000 7eff74800700 1 -- 172.21.15.46:0/2488393823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 msgr2=0x7eff6c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.439+0000 7eff74800700 1 --2- 172.21.15.46:0/2488393823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 0x7eff6c101dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009990 tx=0x7eff5c009ca0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.439+0000 7eff74800700 1 -- 172.21.15.46:0/2488393823 shutdown_connections 2024-04-17T07:56:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.439+0000 7eff74800700 1 --2- 172.21.15.46:0/2488393823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff6c102310 0x7eff6c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.439+0000 7eff74800700 1 --2- 172.21.15.46:0/2488393823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c1079a0 0x7eff6c107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.439+0000 7eff74800700 1 --2- 172.21.15.46:0/2488393823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 0x7eff6c101dd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.439+0000 7eff74800700 1 -- 172.21.15.46:0/2488393823 >> 172.21.15.46:0/2488393823 conn(0x7eff6c078900 msgr2=0x7eff6c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.440+0000 7eff74800700 1 -- 172.21.15.46:0/2488393823 shutdown_connections 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.440+0000 7eff74800700 1 -- 172.21.15.46:0/2488393823 wait complete. 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.441+0000 7eff74800700 1 Processor -- start 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.441+0000 7eff74800700 1 -- start start 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff74800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 0x7eff6c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff74800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff6c102310 0x7eff6c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff74800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c1079a0 0x7eff6c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff74800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c111ca0 con 0x7eff6c1079a0 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff74800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c111b20 con 0x7eff6c101950 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff74800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c111e20 con 0x7eff6c102310 2024-04-17T07:56:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff71e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 0x7eff6c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff72800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c1079a0 0x7eff6c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff71e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 0x7eff6c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54122/0 (socket says 172.21.15.46:54122) 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff71e00700 1 -- 172.21.15.46:0/1809069326 learned_addr learned my addr 172.21.15.46:0/1809069326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.442+0000 7eff71400700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff6c102310 0x7eff6c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff71e00700 1 -- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff6c102310 msgr2=0x7eff6c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff71e00700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff6c102310 0x7eff6c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff71e00700 1 -- 172.21.15.46:0/1809069326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c1079a0 msgr2=0x7eff6c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff71e00700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c1079a0 0x7eff6c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff71e00700 1 -- 172.21.15.46:0/1809069326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c1a7760 con 0x7eff6c101950 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff71400700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff6c102310 0x7eff6c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff71e00700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 0x7eff6c197ef0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9a0 tx=0x7eff6000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000ca10 con 0x7eff6c101950 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6000cbb0 con 0x7eff6c101950 2024-04-17T07:56:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c1a7a50 con 0x7eff6c101950 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.443+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60018680 con 0x7eff6c101950 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.444+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a7f90 con 0x7eff6c101950 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.445+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff6c0630e0 con 0x7eff6c101950 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.446+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7eff60010430 con 0x7eff6c101950 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.448+0000 7eff66a00700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7eff540a3e20 0x7eff540a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.448+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7eff600d09c0 con 0x7eff6c101950 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.448+0000 7eff71400700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7eff540a3e20 0x7eff540a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.448+0000 7eff66a00700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] conn(0x7eff540aef80 0x7eff540b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.448+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 --> [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff60015e80 con 0x7eff540aef80 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.448+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7eff600d52a0 con 0x7eff6c101950 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.449+0000 7eff72800700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] conn(0x7eff540aef80 0x7eff540b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.449+0000 7eff71400700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7eff540a3e20 0x7eff540a62e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff6c19a050 tx=0x7eff5c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.449+0000 7eff72800700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] conn(0x7eff540aef80 0x7eff540b1420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.450+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== osd.8 v2:172.21.15.62:6816/520671268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff60015e80 con 0x7eff540aef80 2024-04-17T07:56:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.469+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 --> [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff6c063590 con 0x7eff540aef80 2024-04-17T07:56:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.469+0000 7eff66a00700 1 -- 172.21.15.46:0/1809069326 <== osd.8 v2:172.21.15.62:6816/520671268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7eff6c063590 con 0x7eff540aef80 2024-04-17T07:56:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] conn(0x7eff540aef80 msgr2=0x7eff540b1420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] conn(0x7eff540aef80 0x7eff540b1420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7eff540a3e20 msgr2=0x7eff540a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7eff540a3e20 0x7eff540a62e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7eff6c19a050 tx=0x7eff5c00b260 comp rx=0 tx=0).stop 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 msgr2=0x7eff6c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 --2- 172.21.15.46:0/1809069326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff6c101950 0x7eff6c197ef0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9a0 tx=0x7eff6000ecb0 comp rx=0 tx=0).stop 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff72800700 1 -- 172.21.15.46:0/1809069326 reap_dead start 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 shutdown_connections 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 >> 172.21.15.46:0/1809069326 conn(0x7eff6c078900 msgr2=0x7eff6c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 shutdown_connections 2024-04-17T07:56:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:28.470+0000 7eff74800700 1 -- 172.21.15.46:0/1809069326 wait complete. 2024-04-17T07:56:31.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:30 smithi046 ceph-mon[32038]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-04-17T07:56:31.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:30 smithi053 ceph-mon[36437]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-04-17T07:56:31.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:30 smithi062 ceph-mon[36507]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-04-17T07:56:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.418+0000 7f894fc00700 1 -- 172.21.15.46:0/2573889331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 msgr2=0x7f894810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.418+0000 7f894fc00700 1 --2- 172.21.15.46:0/2573889331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 0x7f894810f920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8940009f90 tx=0x7f894000b330 comp rx=0 tx=0).stop 2024-04-17T07:56:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.419+0000 7f894fc00700 1 -- 172.21.15.46:0/2573889331 shutdown_connections 2024-04-17T07:56:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.419+0000 7f894fc00700 1 --2- 172.21.15.46:0/2573889331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 0x7f894810f920 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.419+0000 7f894fc00700 1 --2- 172.21.15.46:0/2573889331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948108aa0 0x7f8948108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.419+0000 7f894fc00700 1 --2- 172.21.15.46:0/2573889331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f894806b100 0x7f894806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:31.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.419+0000 7f894fc00700 1 -- 172.21.15.46:0/2573889331 >> 172.21.15.46:0/2573889331 conn(0x7f89480786b0 msgr2=0x7f8948078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.419+0000 7f894fc00700 1 -- 172.21.15.46:0/2573889331 shutdown_connections 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.419+0000 7f894fc00700 1 -- 172.21.15.46:0/2573889331 wait complete. 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.420+0000 7f894fc00700 1 Processor -- start 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894fc00700 1 -- start start 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894fc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894806b100 0x7f8948199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894fc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 0x7f89481996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894fc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948108aa0 0x7f894819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894fc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8948112dc0 con 0x7f894806b100 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894fc00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8948112c40 con 0x7f894806bac0 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894fc00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8948112f40 con 0x7f8948108aa0 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.421+0000 7f894d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894806b100 0x7f8948199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894dc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948108aa0 0x7f894819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:31.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894d200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894806b100 0x7f8948199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39900/0 (socket says 172.21.15.46:39900) 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f8947e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 0x7f89481996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894d200700 1 -- 172.21.15.46:0/4199138569 learned_addr learned my addr 172.21.15.46:0/4199138569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894dc00700 1 -- 172.21.15.46:0/4199138569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 msgr2=0x7f89481996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894dc00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 0x7f89481996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894dc00700 1 -- 172.21.15.46:0/4199138569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894806b100 msgr2=0x7f8948199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894dc00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f894806b100 0x7f8948199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894dc00700 1 -- 172.21.15.46:0/4199138569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894819e1a0 con 0x7f8948108aa0 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f8947e00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f894806bac0 0x7f89481996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.422+0000 7f894dc00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948108aa0 0x7f894819da60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8940009f90 tx=0x7f894000cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.423+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940021070 con 0x7f8948108aa0 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.423+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8940007ac0 con 0x7f8948108aa0 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.423+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894819e430 con 0x7f8948108aa0 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.423+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89400052a0 con 0x7f8948108aa0 2024-04-17T07:56:31.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.423+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89481a5d10 con 0x7f8948108aa0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.424+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89480631e0 con 0x7f8948108aa0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.427+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8940024070 con 0x7f8948108aa0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.428+0000 7f8945600700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89300a3d50 0x7f89300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.429+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f894000f020 con 0x7f8948108aa0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.429+0000 7f8945600700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] conn(0x7f89300aeeb0 0x7f89300b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.429+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 --> [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f894001ae50 con 0x7f89300aeeb0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.429+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f89400d2220 con 0x7f8948108aa0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.429+0000 7f894d200700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89300a3d50 0x7f89300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.429+0000 7f8947e00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] conn(0x7f89300aeeb0 0x7f89300b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.430+0000 7f894d200700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89300a3d50 0x7f89300a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f893c005950 tx=0x7f893c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.430+0000 7f8947e00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] conn(0x7f89300aeeb0 0x7f89300b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.431+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== osd.2 v2:172.21.15.62:6800/3705420480 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f894001ae50 con 0x7f89300aeeb0 2024-04-17T07:56:31.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.450+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 --> [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8948063610 con 0x7f89300aeeb0 2024-04-17T07:56:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.451+0000 7f8945600700 1 -- 172.21.15.46:0/4199138569 <== osd.2 v2:172.21.15.62:6800/3705420480 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8948063610 con 0x7f89300aeeb0 2024-04-17T07:56:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.451+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] conn(0x7f89300aeeb0 msgr2=0x7f89300b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.451+0000 7f894fc00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] conn(0x7f89300aeeb0 0x7f89300b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.451+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89300a3d50 msgr2=0x7f89300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894fc00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f89300a3d50 0x7f89300a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f893c005950 tx=0x7f893c00b500 comp rx=0 tx=0).stop 2024-04-17T07:56:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948108aa0 msgr2=0x7f894819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894fc00700 1 --2- 172.21.15.46:0/4199138569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948108aa0 0x7f894819da60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8940009f90 tx=0x7f894000cfa0 comp rx=0 tx=0).stop 2024-04-17T07:56:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894dc00700 1 -- 172.21.15.46:0/4199138569 reap_dead start 2024-04-17T07:56:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 shutdown_connections 2024-04-17T07:56:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 >> 172.21.15.46:0/4199138569 conn(0x7f89480786b0 msgr2=0x7f8948100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 shutdown_connections 2024-04-17T07:56:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:29.452+0000 7f894fc00700 1 -- 172.21.15.46:0/4199138569 wait complete. 2024-04-17T07:56:32.586 INFO:teuthology.orchestra.run.smithi046.stdout:253403070507 2024-04-17T07:56:32.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.025+0000 7fa37ca00700 1 -- 172.21.15.46:0/399142488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378103460 msgr2=0x7fa37810f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.025+0000 7fa37ca00700 1 --2- 172.21.15.46:0/399142488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378103460 0x7fa37810f8c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa37000b0a0 tx=0x7fa37000b3b0 comp rx=0 tx=0).stop 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.025+0000 7fa37ca00700 1 -- 172.21.15.46:0/399142488 shutdown_connections 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.025+0000 7fa37ca00700 1 --2- 172.21.15.46:0/399142488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378103460 0x7fa37810f8c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.025+0000 7fa37ca00700 1 --2- 172.21.15.46:0/399142488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa378108a60 0x7fa378108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.025+0000 7fa37ca00700 1 --2- 172.21.15.46:0/399142488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa378102aa0 0x7fa378102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.025+0000 7fa37ca00700 1 -- 172.21.15.46:0/399142488 >> 172.21.15.46:0/399142488 conn(0x7fa378079bb0 msgr2=0x7fa378079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.026+0000 7fa37ca00700 1 -- 172.21.15.46:0/399142488 shutdown_connections 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.026+0000 7fa37ca00700 1 -- 172.21.15.46:0/399142488 wait complete. 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.027+0000 7fa37ca00700 1 Processor -- start 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.027+0000 7fa37ca00700 1 -- start start 2024-04-17T07:56:33.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.027+0000 7fa37ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378102aa0 0x7fa378074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.027+0000 7fa37ca00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa378103460 0x7fa378075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.027+0000 7fa37ca00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa378108a60 0x7fa3780757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa37ca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa378112d90 con 0x7fa378102aa0 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa37ca00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa378112c10 con 0x7fa378103460 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa37ca00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa378112f10 con 0x7fa378108a60 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378102aa0 0x7fa378074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378102aa0 0x7fa378074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39918/0 (socket says 172.21.15.46:39918) 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375e00700 1 -- 172.21.15.46:0/3631210092 learned_addr learned my addr 172.21.15.46:0/3631210092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375400700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa378103460 0x7fa378075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa376800700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa378108a60 0x7fa3780757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375400700 1 -- 172.21.15.46:0/3631210092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa378108a60 msgr2=0x7fa3780757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375400700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa378108a60 0x7fa3780757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375400700 1 -- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378102aa0 msgr2=0x7fa378074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375400700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa378102aa0 0x7fa378074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa375400700 1 -- 172.21.15.46:0/3631210092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3781a7730 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.028+0000 7fa376800700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa378108a60 0x7fa3780757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.029+0000 7fa375400700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa378103460 0x7fa378075290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e9f0 tx=0x7fa36400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.029+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36400c9c0 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.029+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36400cb60 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.029+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3781a7a20 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.029+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364018680 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.029+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3781a7f60 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.030+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa378063120 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.032+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa364010430 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.033+0000 7fa36ea00700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa35c0a3cd0 0x7fa35c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.034+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fa3640d0a30 con 0x7fa378103460 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.034+0000 7fa375e00700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa35c0a3cd0 0x7fa35c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.034+0000 7fa36ea00700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] conn(0x7fa35c0aee30 0x7fa35c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.034+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 --> [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa364015e80 con 0x7fa35c0aee30 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.034+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa3640d52a0 con 0x7fa378103460 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.034+0000 7fa376800700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] conn(0x7fa35c0aee30 0x7fa35c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.034+0000 7fa375e00700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa35c0a3cd0 0x7fa35c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa368007c10 tx=0x7fa368007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.035+0000 7fa376800700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] conn(0x7fa35c0aee30 0x7fa35c0b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.036+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== osd.3 v2:172.21.15.46:6810/24338866 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa364015e80 con 0x7fa35c0aee30 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.059+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 --> [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3780635a0 con 0x7fa35c0aee30 2024-04-17T07:56:33.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.060+0000 7fa36ea00700 1 -- 172.21.15.46:0/3631210092 <== osd.3 v2:172.21.15.46:6810/24338866 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa3780635a0 con 0x7fa35c0aee30 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.060+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] conn(0x7fa35c0aee30 msgr2=0x7fa35c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.060+0000 7fa37ca00700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] conn(0x7fa35c0aee30 0x7fa35c0b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.060+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa35c0a3cd0 msgr2=0x7fa35c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.060+0000 7fa37ca00700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa35c0a3cd0 0x7fa35c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa368007c10 tx=0x7fa368007480 comp rx=0 tx=0).stop 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.060+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa378103460 msgr2=0x7fa378075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.060+0000 7fa37ca00700 1 --2- 172.21.15.46:0/3631210092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa378103460 0x7fa378075290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e9f0 tx=0x7fa36400ed00 comp rx=0 tx=0).stop 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.061+0000 7fa376800700 1 -- 172.21.15.46:0/3631210092 reap_dead start 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.061+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 shutdown_connections 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.061+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 >> 172.21.15.46:0/3631210092 conn(0x7fa378079bb0 msgr2=0x7fa378101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.061+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 shutdown_connections 2024-04-17T07:56:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:31.061+0000 7fa37ca00700 1 -- 172.21.15.46:0/3631210092 wait complete. 2024-04-17T07:56:33.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:32 smithi046 ceph-mon[32038]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-04-17T07:56:33.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:32 smithi053 ceph-mon[36437]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-04-17T07:56:33.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:32 smithi062 ceph-mon[36507]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-04-17T07:56:34.288 INFO:teuthology.orchestra.run.smithi046.stdout:107374182472 2024-04-17T07:56:34.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-17T07:56:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 -- 172.21.15.46:0/3005896523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854107950 msgr2=0x7f7854107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 --2- 172.21.15.46:0/3005896523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854107950 0x7f7854107d30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f78440099c0 tx=0x7f7844009cd0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 -- 172.21.15.46:0/3005896523 shutdown_connections 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 --2- 172.21.15.46:0/3005896523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78541034c0 0x7f785410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 --2- 172.21.15.46:0/3005896523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854107950 0x7f7854107d30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 --2- 172.21.15.46:0/3005896523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854102b00 0x7f7854102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 -- 172.21.15.46:0/3005896523 >> 172.21.15.46:0/3005896523 conn(0x7f7854079bb0 msgr2=0x7f7854079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 -- 172.21.15.46:0/3005896523 shutdown_connections 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.310+0000 7f7859e00700 1 -- 172.21.15.46:0/3005896523 wait complete. 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.311+0000 7f7859e00700 1 Processor -- start 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7859e00700 1 -- start start 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7859e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854102b00 0x7f785419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7859e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78541034c0 0x7f785419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7859e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854107950 0x7f785419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7859e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7854112da0 con 0x7f7854102b00 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7859e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7854112c20 con 0x7f78541034c0 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7859e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7854112f20 con 0x7f7854107950 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7852000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78541034c0 0x7f785419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.312+0000 7f7852a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854102b00 0x7f785419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78541034c0 0x7f785419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54182/0 (socket says 172.21.15.46:54182) 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854102b00 0x7f785419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39960/0 (socket says 172.21.15.46:39960) 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 -- 172.21.15.46:0/3080965940 learned_addr learned my addr 172.21.15.46:0/3080965940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7853400700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854107950 0x7f785419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 -- 172.21.15.46:0/3080965940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854107950 msgr2=0x7f785419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854107950 0x7f785419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 -- 172.21.15.46:0/3080965940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854102b00 msgr2=0x7f785419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7854102b00 0x7f785419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 -- 172.21.15.46:0/3080965940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78541a0470 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.313+0000 7f7852000700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78541034c0 0x7f785419b9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f784000e9a0 tx=0x7f784000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.314+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784000ca10 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.314+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784000cbb0 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.314+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840018680 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.314+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78541a0760 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.314+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78541a7fe0 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.315+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78541012d0 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.317+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7840010430 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.319+0000 7f7858a00700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78380a3d50 0x7f78380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.319+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f78400d12f0 con 0x7f78541034c0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.319+0000 7f7852a00700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78380a3d50 0x7f78380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.319+0000 7f7858a00700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] conn(0x7f78380aeeb0 0x7f78380b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.319+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 --> [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78380b1a20 con 0x7f78380aeeb0 2024-04-17T07:56:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.319+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f78400d16a0 con 0x7f78541034c0 2024-04-17T07:56:34.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.320+0000 7f7852a00700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78380a3d50 0x7f78380a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7844000c00 tx=0x7f7844005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:34.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.320+0000 7f7853400700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] conn(0x7f78380aeeb0 0x7f78380b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:34.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.321+0000 7f7853400700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] conn(0x7f78380aeeb0 0x7f78380b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.322+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== osd.4 v2:172.21.15.53:6808/3909086430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78380b1a20 con 0x7f78380aeeb0 2024-04-17T07:56:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.340+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 --> [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78541041c0 con 0x7f78380aeeb0 2024-04-17T07:56:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.341+0000 7f7858a00700 1 -- 172.21.15.46:0/3080965940 <== osd.4 v2:172.21.15.53:6808/3909086430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f78541041c0 con 0x7f78380aeeb0 2024-04-17T07:56:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] conn(0x7f78380aeeb0 msgr2=0x7f78380b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] conn(0x7f78380aeeb0 0x7f78380b1350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78380a3d50 msgr2=0x7f78380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78380a3d50 0x7f78380a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7844000c00 tx=0x7f7844005fb0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78541034c0 msgr2=0x7f785419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 --2- 172.21.15.46:0/3080965940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78541034c0 0x7f785419b9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f784000e9a0 tx=0x7f784000ecb0 comp rx=0 tx=0).stop 2024-04-17T07:56:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7853400700 1 -- 172.21.15.46:0/3080965940 reap_dead start 2024-04-17T07:56:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 shutdown_connections 2024-04-17T07:56:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 >> 172.21.15.46:0/3080965940 conn(0x7f7854079bb0 msgr2=0x7f7854100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 shutdown_connections 2024-04-17T07:56:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:34.342+0000 7f7859e00700 1 -- 172.21.15.46:0/3080965940 wait complete. 2024-04-17T07:56:34.738 INFO:teuthology.orchestra.run.smithi046.stdout:309237645343 2024-04-17T07:56:34.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-17T07:56:34.992 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:34 smithi046 ceph-mon[32038]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-04-17T07:56:35.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:34 smithi053 ceph-mon[36437]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.703+0000 7fdb3f200700 1 -- 172.21.15.46:0/2040797403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38074e80 msgr2=0x7fdb38075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.703+0000 7fdb3f200700 1 --2- 172.21.15.46:0/2040797403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38074e80 0x7fdb38075260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009990 tx=0x7fdb2c009ca0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.703+0000 7fdb3f200700 1 -- 172.21.15.46:0/2040797403 shutdown_connections 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.703+0000 7fdb3f200700 1 --2- 172.21.15.46:0/2040797403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb3810f1c0 0x7fdb38111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.703+0000 7fdb3f200700 1 --2- 172.21.15.46:0/2040797403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38074e80 0x7fdb38075260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.703+0000 7fdb3f200700 1 --2- 172.21.15.46:0/2040797403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb380757a0 0x7fdb3810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.703+0000 7fdb3f200700 1 -- 172.21.15.46:0/2040797403 >> 172.21.15.46:0/2040797403 conn(0x7fdb380fe4d0 msgr2=0x7fdb381008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.704+0000 7fdb3f200700 1 -- 172.21.15.46:0/2040797403 shutdown_connections 2024-04-17T07:56:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.704+0000 7fdb3f200700 1 -- 172.21.15.46:0/2040797403 wait complete. 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 Processor -- start 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 -- start start 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb38074e80 0x7fdb3819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb380757a0 0x7fdb3819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb3810f1c0 0x7fdb3819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38114ee0 con 0x7fdb380757a0 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38114d60 con 0x7fdb3810f1c0 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.705+0000 7fdb3f200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38115060 con 0x7fdb38074e80 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb380757a0 0x7fdb3819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb380757a0 0x7fdb3819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39944/0 (socket says 172.21.15.46:39944) 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3de00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb38074e80 0x7fdb3819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 -- 172.21.15.46:0/4068455526 learned_addr learned my addr 172.21.15.46:0/4068455526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3e800700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb3810f1c0 0x7fdb3819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 -- 172.21.15.46:0/4068455526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb38074e80 msgr2=0x7fdb3819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb38074e80 0x7fdb3819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 -- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb3810f1c0 msgr2=0x7fdb3819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb3810f1c0 0x7fdb3819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3d400700 1 -- 172.21.15.46:0/4068455526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb381a0310 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.706+0000 7fdb3de00700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb38074e80 0x7fdb3819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.707+0000 7fdb3d400700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb380757a0 0x7fdb3819b890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800e9a0 tx=0x7fdb2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.707+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2800ca10 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.707+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2800cbb0 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.707+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb381a0600 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.707+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28018680 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.707+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381a7e80 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.708+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb38063160 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.711+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb28010430 con 0x7fdb380757a0 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.713+0000 7fdb32a00700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdb200a3cd0 0x7fdb200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.713+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fdb280d1380 con 0x7fdb380757a0 2024-04-17T07:56:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.713+0000 7fdb3de00700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdb200a3cd0 0x7fdb200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.713+0000 7fdb32a00700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] conn(0x7fdb200aee30 0x7fdb200b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.713+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 --> [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb28017e70 con 0x7fdb200aee30 2024-04-17T07:56:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.713+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fdb280d1730 con 0x7fdb380757a0 2024-04-17T07:56:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.714+0000 7fdb3e800700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] conn(0x7fdb200aee30 0x7fdb200b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.714+0000 7fdb3de00700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdb200a3cd0 0x7fdb200a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c006010 tx=0x7fdb2c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.714+0000 7fdb3e800700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] conn(0x7fdb200aee30 0x7fdb200b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.715+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== osd.7 v2:172.21.15.53:6816/3581494235 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb28017e70 con 0x7fdb200aee30 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.734+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 --> [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb380635e0 con 0x7fdb200aee30 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb32a00700 1 -- 172.21.15.46:0/4068455526 <== osd.7 v2:172.21.15.53:6816/3581494235 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb380635e0 con 0x7fdb200aee30 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] conn(0x7fdb200aee30 msgr2=0x7fdb200b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] conn(0x7fdb200aee30 0x7fdb200b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdb200a3cd0 msgr2=0x7fdb200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdb200a3cd0 0x7fdb200a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c006010 tx=0x7fdb2c00b260 comp rx=0 tx=0).stop 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb380757a0 msgr2=0x7fdb3819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 --2- 172.21.15.46:0/4068455526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb380757a0 0x7fdb3819b890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800e9a0 tx=0x7fdb2800ecb0 comp rx=0 tx=0).stop 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3e800700 1 -- 172.21.15.46:0/4068455526 reap_dead start 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 shutdown_connections 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.735+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 >> 172.21.15.46:0/4068455526 conn(0x7fdb380fe4d0 msgr2=0x7fdb380ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.736+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 shutdown_connections 2024-04-17T07:56:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:33.736+0000 7fdb3f200700 1 -- 172.21.15.46:0/4068455526 wait complete. 2024-04-17T07:56:35.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:34 smithi062 ceph-mon[36507]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 -- 172.21.15.46:0/3632642466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d8108a60 msgr2=0x7f60d8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 --2- 172.21.15.46:0/3632642466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d8108a60 0x7f60d8108e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009990 tx=0x7f60c8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 -- 172.21.15.46:0/3632642466 shutdown_connections 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 --2- 172.21.15.46:0/3632642466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d8103460 0x7f60d810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 --2- 172.21.15.46:0/3632642466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d8108a60 0x7f60d8108e40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 --2- 172.21.15.46:0/3632642466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60d8102aa0 0x7f60d8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 -- 172.21.15.46:0/3632642466 >> 172.21.15.46:0/3632642466 conn(0x7f60d8079bb0 msgr2=0x7f60d8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.065+0000 7f60dec00700 1 -- 172.21.15.46:0/3632642466 shutdown_connections 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.066+0000 7f60dec00700 1 -- 172.21.15.46:0/3632642466 wait complete. 2024-04-17T07:56:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.067+0000 7f60dec00700 1 Processor -- start 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.067+0000 7f60dec00700 1 -- start start 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dec00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d8102aa0 0x7f60d8074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dec00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d8103460 0x7f60d8075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dec00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60d8108a60 0x7f60d80757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dec00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112d50 con 0x7f60d8103460 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dec00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112bd0 con 0x7f60d8108a60 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dec00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112ed0 con 0x7f60d8102aa0 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dcc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60d8108a60 0x7f60d80757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60d7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d8103460 0x7f60d8075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dcc00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60d8108a60 0x7f60d80757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:54204/0 (socket says 172.21.15.46:54204) 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60dcc00700 1 -- 172.21.15.46:0/3451457629 learned_addr learned my addr 172.21.15.46:0/3451457629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.068+0000 7f60d7e00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d8102aa0 0x7f60d8074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60dcc00700 1 -- 172.21.15.46:0/3451457629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d8102aa0 msgr2=0x7f60d8074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60dcc00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60d8102aa0 0x7f60d8074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60dcc00700 1 -- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d8103460 msgr2=0x7f60d8075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60dcc00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d8103460 0x7f60d8075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60dcc00700 1 -- 172.21.15.46:0/3451457629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d81a77f0 con 0x7f60d8108a60 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60dcc00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60d8108a60 0x7f60d80757d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60d000ef30 tx=0x7f60d000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d00092e0 con 0x7f60d8108a60 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d0004bb0 con 0x7f60d8108a60 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.069+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004280 con 0x7f60d8108a60 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.070+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d81a7a80 con 0x7f60d8108a60 2024-04-17T07:56:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.070+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d81a7f70 con 0x7f60d8108a60 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.071+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60d8063120 con 0x7f60d8108a60 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.073+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f60d001b030 con 0x7f60d8108a60 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.074+0000 7f60d4c00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f60bc0a3cd0 0x7f60bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.075+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f60d00d04d0 con 0x7f60d8108a60 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.075+0000 7f60d7e00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f60bc0a3cd0 0x7f60bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.075+0000 7f60d4c00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] conn(0x7f60bc0aee30 0x7f60bc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.075+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 --> [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60d001fe50 con 0x7f60bc0aee30 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.075+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f60d00d0880 con 0x7f60d8108a60 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.075+0000 7f60d7400700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] conn(0x7f60bc0aee30 0x7f60bc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.076+0000 7f60d7e00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f60bc0a3cd0 0x7f60bc0a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098e0 tx=0x7f60c8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.076+0000 7f60d7400700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] conn(0x7f60bc0aee30 0x7f60bc0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.077+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== osd.0 v2:172.21.15.46:6802/3982963402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60d001fe50 con 0x7f60bc0aee30 2024-04-17T07:56:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.096+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 --> [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60d8063520 con 0x7f60bc0aee30 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.096+0000 7f60d4c00700 1 -- 172.21.15.46:0/3451457629 <== osd.0 v2:172.21.15.46:6802/3982963402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f60d8063520 con 0x7f60bc0aee30 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.097+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] conn(0x7f60bc0aee30 msgr2=0x7f60bc0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.097+0000 7f60dec00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] conn(0x7f60bc0aee30 0x7f60bc0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.097+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f60bc0a3cd0 msgr2=0x7f60bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.097+0000 7f60dec00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f60bc0a3cd0 0x7f60bc0a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098e0 tx=0x7f60c8005960 comp rx=0 tx=0).stop 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.097+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60d8108a60 msgr2=0x7f60d80757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.097+0000 7f60dec00700 1 --2- 172.21.15.46:0/3451457629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60d8108a60 0x7f60d80757d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60d000ef30 tx=0x7f60d000c3a0 comp rx=0 tx=0).stop 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.097+0000 7f60dcc00700 1 -- 172.21.15.46:0/3451457629 reap_dead start 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.098+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 shutdown_connections 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.098+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 >> 172.21.15.46:0/3451457629 conn(0x7f60d8079bb0 msgr2=0x7f60d81018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.098+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 shutdown_connections 2024-04-17T07:56:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:36.098+0000 7f60dec00700 1 -- 172.21.15.46:0/3451457629 wait complete. 2024-04-17T07:56:37.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:36 smithi046 ceph-mon[32038]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:37.122 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:36 smithi062 ceph-mon[36507]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:37.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:36 smithi053 ceph-mon[36437]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:38.226 INFO:teuthology.orchestra.run.smithi046.stdout:77309411408 2024-04-17T07:56:38.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-17T07:56:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 -- 172.21.15.46:0/4094058303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c108a40 msgr2=0x7f3d3c108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 --2- 172.21.15.46:0/4094058303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c108a40 0x7f3d3c108e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009990 tx=0x7f3d30009ca0 comp rx=0 tx=0).stop 2024-04-17T07:56:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 -- 172.21.15.46:0/4094058303 shutdown_connections 2024-04-17T07:56:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 --2- 172.21.15.46:0/4094058303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3c06b100 0x7f3d3c10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 --2- 172.21.15.46:0/4094058303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c108a40 0x7f3d3c108e20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 --2- 172.21.15.46:0/4094058303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3c06a760 0x7f3d3c06abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 -- 172.21.15.46:0/4094058303 >> 172.21.15.46:0/4094058303 conn(0x7f3d3c078900 msgr2=0x7f3d3c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 -- 172.21.15.46:0/4094058303 shutdown_connections 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.456+0000 7f3d44000700 1 -- 172.21.15.46:0/4094058303 wait complete. 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.457+0000 7f3d44000700 1 Processor -- start 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.457+0000 7f3d44000700 1 -- start start 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d44000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3c06a760 0x7f3d3c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d44000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3c06b100 0x7f3d3c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d44000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c108a40 0x7f3d3c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d44000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c112d70 con 0x7f3d3c06b100 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d44000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c112bf0 con 0x7f3d3c06a760 2024-04-17T07:56:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d44000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c112ef0 con 0x7f3d3c108a40 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d40c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3c06b100 0x7f3d3c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d40c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3c06b100 0x7f3d3c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51940/0 (socket says 172.21.15.46:51940) 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d40c00700 1 -- 172.21.15.46:0/3987340283 learned_addr learned my addr 172.21.15.46:0/3987340283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d41600700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3c06a760 0x7f3d3c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d42000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c108a40 0x7f3d3c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d40c00700 1 -- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c108a40 msgr2=0x7f3d3c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.458+0000 7f3d40c00700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c108a40 0x7f3d3c19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d40c00700 1 -- 172.21.15.46:0/3987340283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3c06a760 msgr2=0x7f3d3c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d40c00700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3c06a760 0x7f3d3c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d40c00700 1 -- 172.21.15.46:0/3987340283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c19e220 con 0x7f3d3c06b100 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d41600700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d3c06a760 0x7f3d3c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d40c00700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3c06b100 0x7f3d3c1997a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800d980 tx=0x7f3d2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28016070 con 0x7f3d3c06b100 2024-04-17T07:56:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d28004d10 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28005670 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c19e510 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.459+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1a5d90 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.460+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d3c0631a0 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.463+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d28004750 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.464+0000 7f3d3a000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3d240a3cd0 0x7f3d240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.465+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3d280d0dc0 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.465+0000 7f3d41600700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3d240a3cd0 0x7f3d240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.465+0000 7f3d3a000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] conn(0x7f3d240aee30 0x7f3d240b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.465+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 --> [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d28015ec0 con 0x7f3d240aee30 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.465+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3d280d1170 con 0x7f3d3c06b100 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.465+0000 7f3d42000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] conn(0x7f3d240aee30 0x7f3d240b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.466+0000 7f3d41600700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3d240a3cd0 0x7f3d240a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d30006010 tx=0x7f3d3000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.466+0000 7f3d42000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] conn(0x7f3d240aee30 0x7f3d240b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.467+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== osd.11 v2:172.21.15.62:6824/359287884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d28015ec0 con 0x7f3d240aee30 2024-04-17T07:56:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.486+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 --> [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d3c063620 con 0x7f3d240aee30 2024-04-17T07:56:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d3a000700 1 -- 172.21.15.46:0/3987340283 <== osd.11 v2:172.21.15.62:6824/359287884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d3c063620 con 0x7f3d240aee30 2024-04-17T07:56:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] conn(0x7f3d240aee30 msgr2=0x7f3d240b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d44000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] conn(0x7f3d240aee30 0x7f3d240b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3d240a3cd0 msgr2=0x7f3d240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d44000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3d240a3cd0 0x7f3d240a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d30006010 tx=0x7f3d3000b260 comp rx=0 tx=0).stop 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3c06b100 msgr2=0x7f3d3c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d44000700 1 --2- 172.21.15.46:0/3987340283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d3c06b100 0x7f3d3c1997a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800d980 tx=0x7f3d2800dc90 comp rx=0 tx=0).stop 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.487+0000 7f3d42000700 1 -- 172.21.15.46:0/3987340283 reap_dead start 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.488+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 shutdown_connections 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.488+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 >> 172.21.15.46:0/3987340283 conn(0x7f3d3c078900 msgr2=0x7f3d3c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.488+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 shutdown_connections 2024-04-17T07:56:38.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:38.488+0000 7f3d44000700 1 -- 172.21.15.46:0/3987340283 wait complete. 2024-04-17T07:56:39.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:38 smithi053 ceph-mon[36437]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:39.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:38 smithi062 ceph-mon[36507]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:39.333 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:38 smithi046 ceph-mon[32038]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:40.056 INFO:teuthology.orchestra.run.smithi046.stdout:141733920836 2024-04-17T07:56:40.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-17T07:56:40.162 INFO:teuthology.orchestra.run.smithi046.stdout:34359738457 2024-04-17T07:56:40.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.383+0000 7ff107000700 1 -- 172.21.15.46:0/595206271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1001079a0 msgr2=0x7ff100107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.383+0000 7ff107000700 1 --2- 172.21.15.46:0/595206271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1001079a0 0x7ff100107d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00098f0 tx=0x7ff0f0009c00 comp rx=0 tx=0).stop 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.384+0000 7ff107000700 1 -- 172.21.15.46:0/595206271 shutdown_connections 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.384+0000 7ff107000700 1 --2- 172.21.15.46:0/595206271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff100102310 0x7ff10010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.384+0000 7ff107000700 1 --2- 172.21.15.46:0/595206271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1001079a0 0x7ff100107d80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.384+0000 7ff107000700 1 --2- 172.21.15.46:0/595206271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff100101950 0x7ff100101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.384+0000 7ff107000700 1 -- 172.21.15.46:0/595206271 >> 172.21.15.46:0/595206271 conn(0x7ff100078900 msgr2=0x7ff100078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.384+0000 7ff107000700 1 -- 172.21.15.46:0/595206271 shutdown_connections 2024-04-17T07:56:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.384+0000 7ff107000700 1 -- 172.21.15.46:0/595206271 wait complete. 2024-04-17T07:56:40.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.385+0000 7ff107000700 1 Processor -- start 2024-04-17T07:56:40.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.386+0000 7ff107000700 1 -- start start 2024-04-17T07:56:40.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.386+0000 7ff107000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff100101950 0x7ff100197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.386+0000 7ff107000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff100102310 0x7ff1001983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.386+0000 7ff107000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1001079a0 0x7ff10019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.386+0000 7ff107000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff100111c90 con 0x7ff100101950 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.386+0000 7ff107000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff100111b10 con 0x7ff1001079a0 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.386+0000 7ff107000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff100111e10 con 0x7ff100102310 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.387+0000 7ff0ffe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff100101950 0x7ff100197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.387+0000 7ff0ffe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff100101950 0x7ff100197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51956/0 (socket says 172.21.15.46:51956) 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.387+0000 7ff0ffe00700 1 -- 172.21.15.46:0/4042040995 learned_addr learned my addr 172.21.15.46:0/4042040995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.387+0000 7ff0ff400700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff100102310 0x7ff1001983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.387+0000 7ff0ffe00700 1 -- 172.21.15.46:0/4042040995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff100102310 msgr2=0x7ff1001983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.387+0000 7ff0ffe00700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff100102310 0x7ff1001983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.387+0000 7ff0ffe00700 1 -- 172.21.15.46:0/4042040995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1001079a0 msgr2=0x7ff10019cef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:56:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.388+0000 7ff0ffe00700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1001079a0 0x7ff10019cef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.388+0000 7ff0ffe00700 1 -- 172.21.15.46:0/4042040995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10019d430 con 0x7ff100101950 2024-04-17T07:56:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.388+0000 7ff0ff400700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff100102310 0x7ff1001983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.388+0000 7ff0ffe00700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff100101950 0x7ff100197ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005950 tx=0x7ff0f0005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.389+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00052e0 con 0x7ff100101950 2024-04-17T07:56:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.389+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f000ba40 con 0x7ff100101950 2024-04-17T07:56:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.389+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10019d6c0 con 0x7ff100101950 2024-04-17T07:56:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.390+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0003bb0 con 0x7ff100101950 2024-04-17T07:56:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.390+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10019dba0 con 0x7ff100101950 2024-04-17T07:56:40.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.391+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff10019e090 con 0x7ff100101950 2024-04-17T07:56:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.393+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff0f0005480 con 0x7ff100101950 2024-04-17T07:56:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.395+0000 7ff0fcc00700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0e40a3e20 0x7ff0e40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.396+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff0f00d1cc0 con 0x7ff100101950 2024-04-17T07:56:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.396+0000 7ff0ff400700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0e40a3e20 0x7ff0e40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.396+0000 7ff0fcc00700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] conn(0x7ff0e40aef80 0x7ff0e40b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.396+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 --> [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0f002de50 con 0x7ff0e40aef80 2024-04-17T07:56:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.396+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff0f00d2070 con 0x7ff100101950 2024-04-17T07:56:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.397+0000 7ff105000700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] conn(0x7ff0e40aef80 0x7ff0e40b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.397+0000 7ff0ff400700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0e40a3e20 0x7ff0e40a62e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff100078fe0 tx=0x7ff0ec009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.398+0000 7ff105000700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] conn(0x7ff0e40aef80 0x7ff0e40b1420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.400+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== osd.9 v2:172.21.15.46:6826/1655693057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0f002de50 con 0x7ff0e40aef80 2024-04-17T07:56:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.418+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 --> [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff100103070 con 0x7ff0e40aef80 2024-04-17T07:56:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.419+0000 7ff0fcc00700 1 -- 172.21.15.46:0/4042040995 <== osd.9 v2:172.21.15.46:6826/1655693057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff100103070 con 0x7ff0e40aef80 2024-04-17T07:56:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] conn(0x7ff0e40aef80 msgr2=0x7ff0e40b1420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] conn(0x7ff0e40aef80 0x7ff0e40b1420 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0e40a3e20 msgr2=0x7ff0e40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff0e40a3e20 0x7ff0e40a62e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff100078fe0 tx=0x7ff0ec009310 comp rx=0 tx=0).stop 2024-04-17T07:56:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff100101950 msgr2=0x7ff100197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 --2- 172.21.15.46:0/4042040995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff100101950 0x7ff100197ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005950 tx=0x7ff0f0005040 comp rx=0 tx=0).stop 2024-04-17T07:56:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff105000700 1 -- 172.21.15.46:0/4042040995 reap_dead start 2024-04-17T07:56:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 shutdown_connections 2024-04-17T07:56:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 >> 172.21.15.46:0/4042040995 conn(0x7ff100078900 msgr2=0x7ff1001007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 shutdown_connections 2024-04-17T07:56:40.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:40.420+0000 7ff107000700 1 -- 172.21.15.46:0/4042040995 wait complete. 2024-04-17T07:56:40.835 INFO:teuthology.orchestra.run.smithi046.stdout:219043332146 2024-04-17T07:56:40.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-17T07:56:41.100 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:40 smithi046 ceph-mon[32038]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:40 smithi053 ceph-mon[36437]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:41.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:40 smithi062 ceph-mon[36507]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-17T07:56:42.367 INFO:teuthology.orchestra.run.smithi046.stdout:287762808872 2024-04-17T07:56:42.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.078+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1481688104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74104ec0 msgr2=0x7f4c741052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.078+0000 7f4c7ac00700 1 --2- 172.21.15.46:0/1481688104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74104ec0 0x7f4c741052a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c640099c0 tx=0x7f4c64009cd0 comp rx=0 tx=0).stop 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.079+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1481688104 shutdown_connections 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.079+0000 7f4c7ac00700 1 --2- 172.21.15.46:0/1481688104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c74109fa0 0x7f4c74111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.079+0000 7f4c7ac00700 1 --2- 172.21.15.46:0/1481688104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74104ec0 0x7f4c741052a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.079+0000 7f4c7ac00700 1 --2- 172.21.15.46:0/1481688104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c74105870 0x7f4c741098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.079+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1481688104 >> 172.21.15.46:0/1481688104 conn(0x7f4c74100750 msgr2=0x7f4c74102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.079+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1481688104 shutdown_connections 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.079+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1481688104 wait complete. 2024-04-17T07:56:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.080+0000 7f4c7ac00700 1 Processor -- start 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.080+0000 7f4c7ac00700 1 -- start start 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c7ac00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c7ac00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74105870 0x7f4c7419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c7ac00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c74109fa0 0x7f4c7419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c7ac00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74114fd0 con 0x7f4c74105870 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c7ac00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74114e50 con 0x7f4c74104ec0 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c7ac00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4c74115150 con 0x7f4c74109fa0 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c73e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c73e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59082/0 (socket says 172.21.15.46:59082) 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c73e00700 1 -- 172.21.15.46:0/1989065166 learned_addr learned my addr 172.21.15.46:0/1989065166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c78c00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c74109fa0 0x7f4c7419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.081+0000 7f4c73400700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74105870 0x7f4c7419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c73e00700 1 -- 172.21.15.46:0/1989065166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c74109fa0 msgr2=0x7f4c7419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c73e00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c74109fa0 0x7f4c7419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c73e00700 1 -- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74105870 msgr2=0x7f4c7419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c73e00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74105870 0x7f4c7419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c73e00700 1 -- 172.21.15.46:0/1989065166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c741a0440 con 0x7f4c74104ec0 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c73400700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74105870 0x7f4c7419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c73e00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c64006010 tx=0x7f4c64003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c640050c0 con 0x7f4c74104ec0 2024-04-17T07:56:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6400f460 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c741a06d0 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.082+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64017650 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.083+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c741a7fb0 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.083+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c74063160 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.086+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4c6400f610 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.087+0000 7f4c70c00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4c580a3d50 0x7f4c580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.087+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4c640d18b0 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.087+0000 7f4c73400700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4c580a3d50 0x7f4c580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.087+0000 7f4c70c00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] conn(0x7f4c580aeeb0 0x7f4c580b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.087+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 --> [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c6400b790 con 0x7f4c580aeeb0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.087+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4c640d1c60 con 0x7f4c74104ec0 2024-04-17T07:56:42.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.088+0000 7f4c78c00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] conn(0x7f4c580aeeb0 0x7f4c580b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.088+0000 7f4c73400700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4c580a3d50 0x7f4c580a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4c74101c70 tx=0x7f4c60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.088+0000 7f4c78c00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] conn(0x7f4c580aeeb0 0x7f4c580b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.090+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== osd.6 v2:172.21.15.46:6818/1575836567 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c6400b790 con 0x7f4c580aeeb0 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.109+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 --> [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4c74063560 con 0x7f4c580aeeb0 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c70c00700 1 -- 172.21.15.46:0/1989065166 <== osd.6 v2:172.21.15.46:6818/1575836567 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4c74063560 con 0x7f4c580aeeb0 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] conn(0x7f4c580aeeb0 msgr2=0x7f4c580b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c7ac00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] conn(0x7f4c580aeeb0 0x7f4c580b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4c580a3d50 msgr2=0x7f4c580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c7ac00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4c580a3d50 0x7f4c580a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4c74101c70 tx=0x7f4c60009260 comp rx=0 tx=0).stop 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c74104ec0 msgr2=0x7f4c7419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c7ac00700 1 --2- 172.21.15.46:0/1989065166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c74104ec0 0x7f4c7419b480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c64006010 tx=0x7f4c64003a90 comp rx=0 tx=0).stop 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.110+0000 7f4c78c00700 1 -- 172.21.15.46:0/1989065166 reap_dead start 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.111+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 shutdown_connections 2024-04-17T07:56:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.111+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 >> 172.21.15.46:0/1989065166 conn(0x7f4c74100750 msgr2=0x7f4c74102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.111+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 shutdown_connections 2024-04-17T07:56:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:42.111+0000 7f4c7ac00700 1 -- 172.21.15.46:0/1989065166 wait complete. 2024-04-17T07:56:43.105 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:42 smithi046 ceph-mon[32038]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:56:43.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:42 smithi053 ceph-mon[36437]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:56:43.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:42 smithi062 ceph-mon[36507]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:56:43.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 -- 172.21.15.46:0/2693327071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c28102310 msgr2=0x7f5c2810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 --2- 172.21.15.46:0/2693327071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c28102310 0x7f5c2810e800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000b0a0 tx=0x7f5c2000b3b0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 -- 172.21.15.46:0/2693327071 shutdown_connections 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 --2- 172.21.15.46:0/2693327071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c28102310 0x7f5c2810e800 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 --2- 172.21.15.46:0/2693327071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c281079a0 0x7f5c28107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 --2- 172.21.15.46:0/2693327071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c28101950 0x7f5c28101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 -- 172.21.15.46:0/2693327071 >> 172.21.15.46:0/2693327071 conn(0x7f5c28078900 msgr2=0x7f5c28078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 -- 172.21.15.46:0/2693327071 shutdown_connections 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.404+0000 7f5c2d600700 1 -- 172.21.15.46:0/2693327071 wait complete. 2024-04-17T07:56:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.405+0000 7f5c2d600700 1 Processor -- start 2024-04-17T07:56:43.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.406+0000 7f5c2d600700 1 -- start start 2024-04-17T07:56:43.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.406+0000 7f5c2d600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c28101950 0x7f5c28199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:43.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.406+0000 7f5c2d600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c28102310 0x7f5c281996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:43.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.406+0000 7f5c2d600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c281079a0 0x7f5c2819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:43.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.406+0000 7f5c2d600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28111c30 con 0x7f5c28102310 2024-04-17T07:56:43.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.406+0000 7f5c2d600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28111ab0 con 0x7f5c281079a0 2024-04-17T07:56:43.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.406+0000 7f5c2d600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28111db0 con 0x7f5c28101950 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c28101950 0x7f5c28199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c28101950 0x7f5c28199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:46234/0 (socket says 172.21.15.46:46234) 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 -- 172.21.15.46:0/1510769596 learned_addr learned my addr 172.21.15.46:0/1510769596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26000700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c28102310 0x7f5c281996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c27400700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c281079a0 0x7f5c2819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 -- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c281079a0 msgr2=0x7f5c2819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c281079a0 0x7f5c2819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 -- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c28102310 msgr2=0x7f5c281996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c28102310 0x7f5c281996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.407+0000 7f5c26a00700 1 -- 172.21.15.46:0/1510769596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2819e190 con 0x7f5c28101950 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.408+0000 7f5c26a00700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c28101950 0x7f5c28199180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9a0 tx=0x7f5c1800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.408+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800cbd0 con 0x7f5c28101950 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.408+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1800cd70 con 0x7f5c28101950 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.408+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2819e480 con 0x7f5c28101950 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.408+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18018750 con 0x7f5c28101950 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.408+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c281a5d00 con 0x7f5c28101950 2024-04-17T07:56:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.409+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c280631e0 con 0x7f5c28101950 2024-04-17T07:56:43.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.411+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5c18010430 con 0x7f5c28101950 2024-04-17T07:56:43.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.413+0000 7f5c1f400700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5c0c0a3cd0 0x7f5c0c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:43.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.413+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5c180d1390 con 0x7f5c28101950 2024-04-17T07:56:43.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.413+0000 7f5c26000700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5c0c0a3cd0 0x7f5c0c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:43.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.413+0000 7f5c1f400700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] conn(0x7f5c0c0aee30 0x7f5c0c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:43.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.413+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 --> [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c0c0b19a0 con 0x7f5c0c0aee30 2024-04-17T07:56:43.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.413+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f5c180d1740 con 0x7f5c28101950 2024-04-17T07:56:43.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.414+0000 7f5c27400700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] conn(0x7f5c0c0aee30 0x7f5c0c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:43.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.414+0000 7f5c26000700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5c0c0a3cd0 0x7f5c0c0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c28078fe0 tx=0x7f5c14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:43.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.414+0000 7f5c27400700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] conn(0x7f5c0c0aee30 0x7f5c0c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:43.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.415+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== osd.1 v2:172.21.15.53:6800/445046635 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c0c0b19a0 con 0x7f5c0c0aee30 2024-04-17T07:56:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.433+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 --> [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c28063660 con 0x7f5c0c0aee30 2024-04-17T07:56:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.434+0000 7f5c1f400700 1 -- 172.21.15.46:0/1510769596 <== osd.1 v2:172.21.15.53:6800/445046635 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5c28063660 con 0x7f5c0c0aee30 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] conn(0x7f5c0c0aee30 msgr2=0x7f5c0c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] conn(0x7f5c0c0aee30 0x7f5c0c0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5c0c0a3cd0 msgr2=0x7f5c0c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5c0c0a3cd0 0x7f5c0c0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c28078fe0 tx=0x7f5c14009260 comp rx=0 tx=0).stop 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c28101950 msgr2=0x7f5c28199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 --2- 172.21.15.46:0/1510769596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c28101950 0x7f5c28199180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e9a0 tx=0x7f5c1800ecb0 comp rx=0 tx=0).stop 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c27400700 1 -- 172.21.15.46:0/1510769596 reap_dead start 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 shutdown_connections 2024-04-17T07:56:43.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 >> 172.21.15.46:0/1510769596 conn(0x7f5c28078900 msgr2=0x7f5c2806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:43.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 shutdown_connections 2024-04-17T07:56:43.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:43.435+0000 7f5c2d600700 1 -- 172.21.15.46:0/1510769596 wait complete. 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 -- 172.21.15.46:0/3091377559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 msgr2=0x7f6c30111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 --2- 172.21.15.46:0/3091377559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 0x7f6c30111b70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800b0d0 tx=0x7f6c2800b3e0 comp rx=0 tx=0).stop 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 -- 172.21.15.46:0/3091377559 shutdown_connections 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 --2- 172.21.15.46:0/3091377559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 0x7f6c30111b70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 --2- 172.21.15.46:0/3091377559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c30104ec0 0x7f6c301052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 --2- 172.21.15.46:0/3091377559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c30105870 0x7f6c301098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 -- 172.21.15.46:0/3091377559 >> 172.21.15.46:0/3091377559 conn(0x7f6c30100750 msgr2=0x7f6c30102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 -- 172.21.15.46:0/3091377559 shutdown_connections 2024-04-17T07:56:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.187+0000 7f6c37e00700 1 -- 172.21.15.46:0/3091377559 wait complete. 2024-04-17T07:56:44.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.188+0000 7f6c37e00700 1 Processor -- start 2024-04-17T07:56:44.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.188+0000 7f6c37e00700 1 -- start start 2024-04-17T07:56:44.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c37e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c30104ec0 0x7f6c3019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:44.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c37e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c30105870 0x7f6c3019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:44.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c37e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 0x7f6c3019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:44.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c37e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c30114fe0 con 0x7f6c30109fa0 2024-04-17T07:56:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c37e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6c30114e60 con 0x7f6c30104ec0 2024-04-17T07:56:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c37e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c30115160 con 0x7f6c30105870 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c34a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c30105870 0x7f6c3019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c35400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c30104ec0 0x7f6c3019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c35e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 0x7f6c3019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c34a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c30105870 0x7f6c3019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:46256/0 (socket says 172.21.15.46:46256) 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c35400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c30104ec0 0x7f6c3019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:59120/0 (socket says 172.21.15.46:59120) 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.189+0000 7f6c34a00700 1 -- 172.21.15.46:0/3953239701 learned_addr learned my addr 172.21.15.46:0/3953239701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c34a00700 1 -- 172.21.15.46:0/3953239701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c30104ec0 msgr2=0x7f6c3019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c34a00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c30104ec0 0x7f6c3019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c34a00700 1 -- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 msgr2=0x7f6c3019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c34a00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 0x7f6c3019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c34a00700 1 -- 172.21.15.46:0/3953239701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c301a03c0 con 0x7f6c30105870 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c35e00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c30109fa0 0x7f6c3019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c34a00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c30105870 0x7f6c3019b8f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00ea30 tx=0x7f6c1c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00cbd0 con 0x7f6c30105870 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c004500 con 0x7f6c30105870 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c003e30 con 0x7f6c30105870 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.190+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c301a06b0 con 0x7f6c30105870 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.191+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c301a7f30 con 0x7f6c30105870 2024-04-17T07:56:44.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.191+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c300631a0 con 0x7f6c30105870 2024-04-17T07:56:44.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.194+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6c1c004000 con 0x7f6c30105870 2024-04-17T07:56:44.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.195+0000 7f6c2e000700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c180a3d50 0x7f6c180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.195+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6c1c0d0ad0 con 0x7f6c30105870 2024-04-17T07:56:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.195+0000 7f6c35400700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c180a3d50 0x7f6c180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.195+0000 7f6c2e000700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] conn(0x7f6c180aeeb0 0x7f6c180b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.195+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 --> [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c180b1a20 con 0x7f6c180aeeb0 2024-04-17T07:56:44.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.195+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c0d0e80 con 0x7f6c30105870 2024-04-17T07:56:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.196+0000 7f6c35e00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] conn(0x7f6c180aeeb0 0x7f6c180b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.196+0000 7f6c35400700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c180a3d50 0x7f6c180a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009c60 tx=0x7f6c24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:44.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.197+0000 7f6c35e00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] conn(0x7f6c180aeeb0 0x7f6c180b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:44.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.198+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== osd.5 v2:172.21.15.62:6808/2113355560 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c180b1a20 con 0x7f6c180aeeb0 2024-04-17T07:56:44.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.215+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 --> [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c30063670 con 0x7f6c180aeeb0 2024-04-17T07:56:44.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.216+0000 7f6c2e000700 1 -- 172.21.15.46:0/3953239701 <== osd.5 v2:172.21.15.62:6808/2113355560 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6c30063670 con 0x7f6c180aeeb0 2024-04-17T07:56:44.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] conn(0x7f6c180aeeb0 msgr2=0x7f6c180b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] conn(0x7f6c180aeeb0 0x7f6c180b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c180a3d50 msgr2=0x7f6c180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c180a3d50 0x7f6c180a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009c60 tx=0x7f6c24009260 comp rx=0 tx=0).stop 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c30105870 msgr2=0x7f6c3019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 --2- 172.21.15.46:0/3953239701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c30105870 0x7f6c3019b8f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00ea30 tx=0x7f6c1c00ed40 comp rx=0 tx=0).stop 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c35e00700 1 -- 172.21.15.46:0/3953239701 reap_dead start 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 shutdown_connections 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 >> 172.21.15.46:0/3953239701 conn(0x7f6c30100750 msgr2=0x7f6c301012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 shutdown_connections 2024-04-17T07:56:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:44.217+0000 7f6c37e00700 1 -- 172.21.15.46:0/3953239701 wait complete. 2024-04-17T07:56:44.662 INFO:teuthology.orchestra.run.smithi046.stdout:197568495674 2024-04-17T07:56:44.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-17T07:56:45.045 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:45.080 INFO:teuthology.orchestra.run.smithi046.stdout:343597383710 2024-04-17T07:56:45.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-17T07:56:45.198 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:44 smithi046 ceph-mon[32038]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-17T07:56:45.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:44 smithi053 ceph-mon[36437]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-17T07:56:45.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:44 smithi062 ceph-mon[36507]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-17T07:56:45.796 INFO:teuthology.orchestra.run.smithi046.stdout:163208757313 2024-04-17T07:56:45.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-17T07:56:47.035 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:47.045 INFO:teuthology.orchestra.run.smithi046.stdout:55834574934 2024-04-17T07:56:47.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-17T07:56:47.219 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:46 smithi046 ceph-mon[32038]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T07:56:47.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:46 smithi053 ceph-mon[36437]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T07:56:47.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:46 smithi062 ceph-mon[36507]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T07:56:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:56:49.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:48 smithi053 ceph-mon[36437]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:56:49.244 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:48 smithi046 ceph-mon[32038]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:56:49.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:48 smithi062 ceph-mon[36507]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:56:51.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:50 smithi046 ceph-mon[32038]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:56:51.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:50 smithi053 ceph-mon[36437]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:56:51.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:50 smithi062 ceph-mon[36507]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:56:52.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:52 smithi046 ceph-mon[32038]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:56:53.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:52 smithi053 ceph-mon[36437]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:56:53.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:52 smithi062 ceph-mon[36507]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:56:55.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:54 smithi046 ceph-mon[32038]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:56:55.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:54 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3300605292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T07:56:55.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:54 smithi053 ceph-mon[36437]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:56:55.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:54 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3300605292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T07:56:55.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:54 smithi062 ceph-mon[36507]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:56:55.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:54 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3300605292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 -- 172.21.15.46:0/2825689950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac103b40 msgr2=0x7f9dac103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 --2- 172.21.15.46:0/2825689950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac103b40 0x7f9dac103f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009990 tx=0x7f9da0009ca0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 -- 172.21.15.46:0/2825689950 shutdown_connections 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 --2- 172.21.15.46:0/2825689950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dac10cf70 0x7f9dac10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 --2- 172.21.15.46:0/2825689950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac103b40 0x7f9dac103f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 --2- 172.21.15.46:0/2825689950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dac104460 0x7f9dac10c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 -- 172.21.15.46:0/2825689950 >> 172.21.15.46:0/2825689950 conn(0x7f9dac0fd370 msgr2=0x7f9dac0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 -- 172.21.15.46:0/2825689950 shutdown_connections 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.795+0000 7f9db2a00700 1 -- 172.21.15.46:0/2825689950 wait complete. 2024-04-17T07:56:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.796+0000 7f9db2a00700 1 Processor -- start 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.797+0000 7f9db2a00700 1 -- start start 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.797+0000 7f9db2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dac103b40 0x7f9dac1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.797+0000 7f9db2a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac104460 0x7f9dac199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.797+0000 7f9db2a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dac10cf70 0x7f9dac19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.797+0000 7f9db2a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112ce0 con 0x7f9dac104460 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.797+0000 7f9db2a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112b60 con 0x7f9dac10cf70 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.797+0000 7f9db2a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac112e60 con 0x7f9dac103b40 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac104460 0x7f9dac199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac104460 0x7f9dac199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45040/0 (socket says 172.21.15.46:45040) 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 -- 172.21.15.46:0/3300605292 learned_addr learned my addr 172.21.15.46:0/3300605292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db1600700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dac103b40 0x7f9dac1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db2000700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dac10cf70 0x7f9dac19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 -- 172.21.15.46:0/3300605292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dac103b40 msgr2=0x7f9dac1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dac103b40 0x7f9dac1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 -- 172.21.15.46:0/3300605292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dac10cf70 msgr2=0x7f9dac19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dac10cf70 0x7f9dac19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db0c00700 1 -- 172.21.15.46:0/3300605292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac19e190 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.798+0000 7f9db1600700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dac103b40 0x7f9dac1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.799+0000 7f9db0c00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac104460 0x7f9dac199710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800e9f0 tx=0x7f9d9800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.799+0000 7f9daa000700 1 -- 172.21.15.46:0/3300605292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98004a30 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.799+0000 7f9daa000700 1 -- 172.21.15.46:0/3300605292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9800cca0 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.799+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac19e480 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.799+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1a5d00 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.800+0000 7f9daa000700 1 -- 172.21.15.46:0/3300605292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98005680 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.801+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac19a4e0 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.803+0000 7f9daa000700 1 -- 172.21.15.46:0/3300605292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9d9800c7c0 con 0x7f9dac104460 2024-04-17T07:56:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.806+0000 7f9daa000700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d940a3cd0 0x7f9d940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.806+0000 7f9daa000700 1 -- 172.21.15.46:0/3300605292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9d980d0db0 con 0x7f9dac104460 2024-04-17T07:56:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.806+0000 7f9db1600700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d940a3cd0 0x7f9d940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.807+0000 7f9db1600700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d940a3cd0 0x7f9d940a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9da000b2e0 tx=0x7f9da001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:53.810+0000 7f9daa000700 1 -- 172.21.15.46:0/3300605292 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9d980d5070 con 0x7f9dac104460 2024-04-17T07:56:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.054+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9dac069720 con 0x7f9dac104460 2024-04-17T07:56:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.055+0000 7f9daa000700 1 -- 172.21.15.46:0/3300605292 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d9809d270 con 0x7f9dac104460 2024-04-17T07:56:55.871 INFO:teuthology.orchestra.run.smithi046.stdout:253403070512 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.058+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d940a3cd0 msgr2=0x7f9d940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.058+0000 7f9db2a00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d940a3cd0 0x7f9d940a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9da000b2e0 tx=0x7f9da001a040 comp rx=0 tx=0).stop 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.058+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac104460 msgr2=0x7f9dac199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.058+0000 7f9db2a00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac104460 0x7f9dac199710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800e9f0 tx=0x7f9d9800ed00 comp rx=0 tx=0).stop 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 shutdown_connections 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9d940a3cd0 0x7f9d940a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dac10cf70 0x7f9dac19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dac103b40 0x7f9dac1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 --2- 172.21.15.46:0/3300605292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9dac104460 0x7f9dac199710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 >> 172.21.15.46:0/3300605292 conn(0x7f9dac0fd370 msgr2=0x7f9dac0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 shutdown_connections 2024-04-17T07:56:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:54.059+0000 7f9db2a00700 1 -- 172.21.15.46:0/3300605292 wait complete. 2024-04-17T07:56:56.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070507 got 253403070512 for osd.8 2024-04-17T07:56:56.630 DEBUG:teuthology.parallel:result is None 2024-04-17T07:56:57.212 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:56 smithi046 ceph-mon[32038]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-17T07:56:57.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:56 smithi053 ceph-mon[36437]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-17T07:56:57.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:56 smithi062 ceph-mon[36507]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-17T07:56:59.002 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:58 smithi046 ceph-mon[32038]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:56:59.002 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:56:58 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2946530471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T07:56:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:58 smithi053 ceph-mon[36437]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:56:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:56:58 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2946530471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T07:56:59.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:58 smithi062 ceph-mon[36507]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:56:59.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:56:58 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2946530471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 -- 172.21.15.46:0/3664803421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44109c90 msgr2=0x7f5f4410dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 --2- 172.21.15.46:0/3664803421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44109c90 0x7f5f4410dce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009920 tx=0x7f5f2c009c30 comp rx=0 tx=0).stop 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 -- 172.21.15.46:0/3664803421 shutdown_connections 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 --2- 172.21.15.46:0/3664803421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f4410e3c0 0x7f5f44115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 --2- 172.21.15.46:0/3664803421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f441092e0 0x7f5f441096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 --2- 172.21.15.46:0/3664803421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f44109c90 0x7f5f4410dce0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 -- 172.21.15.46:0/3664803421 >> 172.21.15.46:0/3664803421 conn(0x7f5f44076220 msgr2=0x7f5f44107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:59.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 -- 172.21.15.46:0/3664803421 shutdown_connections 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.169+0000 7f5f49e00700 1 -- 172.21.15.46:0/3664803421 wait complete. 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.171+0000 7f5f49e00700 1 Processor -- start 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.171+0000 7f5f49e00700 1 -- start start 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.171+0000 7f5f49e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f441092e0 0x7f5f441a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.171+0000 7f5f49e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f44109c90 0x7f5f441a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f49e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4410e3c0 0x7f5f4419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f49e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441193d0 con 0x7f5f44109c90 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f49e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44119250 con 0x7f5f441092e0 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f49e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44119550 con 0x7f5f4410e3c0 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f3be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f44109c90 0x7f5f441a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f42a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f441092e0 0x7f5f441a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f3be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f44109c90 0x7f5f441a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53588/0 (socket says 172.21.15.46:53588) 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f3be00700 1 -- 172.21.15.46:0/2946530471 learned_addr learned my addr 172.21.15.46:0/2946530471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:56:59.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.172+0000 7f5f43400700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4410e3c0 0x7f5f4419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f3be00700 1 -- 172.21.15.46:0/2946530471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4410e3c0 msgr2=0x7f5f4419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f3be00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4410e3c0 0x7f5f4419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f3be00700 1 -- 172.21.15.46:0/2946530471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f441092e0 msgr2=0x7f5f441a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f3be00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f441092e0 0x7f5f441a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f3be00700 1 -- 172.21.15.46:0/2946530471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4419fff0 con 0x7f5f44109c90 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f42a00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f441092e0 0x7f5f441a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f3be00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f44109c90 0x7f5f441a5b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00a010 tx=0x7f5f2c00fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f48a00700 1 -- 172.21.15.46:0/2946530471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0051f0 con 0x7f5f44109c90 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f441a0280 con 0x7f5f44109c90 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f48a00700 1 -- 172.21.15.46:0/2946530471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f2c00b560 con 0x7f5f44109c90 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.173+0000 7f5f48a00700 1 -- 172.21.15.46:0/2946530471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c003aa0 con 0x7f5f44109c90 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.174+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f441ac3a0 con 0x7f5f44109c90 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.175+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f4410a7c0 con 0x7f5f44109c90 2024-04-17T07:56:59.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.177+0000 7f5f48a00700 1 -- 172.21.15.46:0/2946530471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5f2c00baa0 con 0x7f5f44109c90 2024-04-17T07:56:59.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.178+0000 7f5f48a00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5f280a3d50 0x7f5f280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:56:59.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.178+0000 7f5f48a00700 1 -- 172.21.15.46:0/2946530471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5f2c0d1890 con 0x7f5f44109c90 2024-04-17T07:56:59.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.179+0000 7f5f42a00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5f280a3d50 0x7f5f280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:56:59.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.179+0000 7f5f42a00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5f280a3d50 0x7f5f280a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005fd0 tx=0x7f5f34009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:56:59.323 INFO:teuthology.orchestra.run.smithi046.stdout:309237645350 2024-04-17T07:56:59.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.183+0000 7f5f48a00700 1 -- 172.21.15.46:0/2946530471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5f2c013030 con 0x7f5f44109c90 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.415+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5f4402df90 con 0x7f5f44109c90 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.416+0000 7f5f48a00700 1 -- 172.21.15.46:0/2946530471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5f2c09dde0 con 0x7f5f44109c90 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5f280a3d50 msgr2=0x7f5f280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5f280a3d50 0x7f5f280a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005fd0 tx=0x7f5f34009310 comp rx=0 tx=0).stop 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f44109c90 msgr2=0x7f5f441a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f44109c90 0x7f5f441a5b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00a010 tx=0x7f5f2c00fb20 comp rx=0 tx=0).stop 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 shutdown_connections 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5f280a3d50 0x7f5f280a6210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5f4410e3c0 0x7f5f4419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f441092e0 0x7f5f441a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 --2- 172.21.15.46:0/2946530471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f44109c90 0x7f5f441a5b30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.419+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 >> 172.21.15.46:0/2946530471 conn(0x7f5f44076220 msgr2=0x7f5f44076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:56:59.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.420+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 shutdown_connections 2024-04-17T07:56:59.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:56:58.421+0000 7f5f49e00700 1 -- 172.21.15.46:0/2946530471 wait complete. 2024-04-17T07:57:00.604 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:01.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645350 for osd.10 2024-04-17T07:57:01.099 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:01.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:00 smithi053 ceph-mon[36437]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:01.241 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:00 smithi046 ceph-mon[32038]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:01.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:00 smithi062 ceph-mon[36507]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:01.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 -- 172.21.15.46:0/3229475252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208010f500 msgr2=0x7f20801118f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 --2- 172.21.15.46:0/3229475252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208010f500 0x7f20801118f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f207800b0a0 tx=0x7f207800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 -- 172.21.15.46:0/3229475252 shutdown_connections 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 --2- 172.21.15.46:0/3229475252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208010f500 0x7f20801118f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 --2- 172.21.15.46:0/3229475252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208006a760 0x7f208006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 --2- 172.21.15.46:0/3229475252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208006b080 0x7f208010efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 -- 172.21.15.46:0/3229475252 >> 172.21.15.46:0/3229475252 conn(0x7f2080078900 msgr2=0x7f2080078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 -- 172.21.15.46:0/3229475252 shutdown_connections 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.872+0000 7f2085800700 1 -- 172.21.15.46:0/3229475252 wait complete. 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.873+0000 7f2085800700 1 Processor -- start 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.873+0000 7f2085800700 1 -- start start 2024-04-17T07:57:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f2085800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208006a760 0x7f208019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f2085800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006b080 0x7f208019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f2085800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208010f500 0x7f208019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f2085800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2080114fa0 con 0x7f208006b080 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f2085800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2080114e20 con 0x7f208006a760 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f2085800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2080115120 con 0x7f208010f500 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f207e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006b080 0x7f208019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f207e000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006b080 0x7f208019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53596/0 (socket says 172.21.15.46:53596) 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f207e000700 1 -- 172.21.15.46:0/2236668984 learned_addr learned my addr 172.21.15.46:0/2236668984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f207f400700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208010f500 0x7f208019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.874+0000 7f207ea00700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208006a760 0x7f208019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f207f400700 1 -- 172.21.15.46:0/2236668984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208006a760 msgr2=0x7f208019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f207f400700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208006a760 0x7f208019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f207f400700 1 -- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006b080 msgr2=0x7f208019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f207f400700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006b080 0x7f208019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f207f400700 1 -- 172.21.15.46:0/2236668984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20801a0410 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f207f400700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208010f500 0x7f208019fcd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f207800b070 tx=0x7f2078004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f2077400700 1 -- 172.21.15.46:0/2236668984 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078021070 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.875+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20801a06a0 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.876+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a7f80 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.876+0000 7f2077400700 1 -- 172.21.15.46:0/2236668984 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2078007b80 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.876+0000 7f2077400700 1 -- 172.21.15.46:0/2236668984 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207802b060 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.877+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2080069740 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.879+0000 7f2077400700 1 -- 172.21.15.46:0/2236668984 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2078024070 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.880+0000 7f2077400700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f20640a3d50 0x7f20640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.880+0000 7f2077400700 1 -- 172.21.15.46:0/2236668984 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f20780d13f0 con 0x7f208010f500 2024-04-17T07:57:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.880+0000 7f207ea00700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f20640a3d50 0x7f20640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:01.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.881+0000 7f207ea00700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f20640a3d50 0x7f20640a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2070009680 tx=0x7f2070009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:01.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:01.885+0000 7f2077400700 1 -- 172.21.15.46:0/2236668984 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2078007d30 con 0x7f208010f500 2024-04-17T07:57:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.124+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f208002df90 con 0x7f208010f500 2024-04-17T07:57:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.125+0000 7f2077400700 1 -- 172.21.15.46:0/2236668984 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f207809d940 con 0x7f208010f500 2024-04-17T07:57:02.125 INFO:teuthology.orchestra.run.smithi046.stdout:219043332151 2024-04-17T07:57:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f20640a3d50 msgr2=0x7f20640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f20640a3d50 0x7f20640a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2070009680 tx=0x7f2070009260 comp rx=0 tx=0).stop 2024-04-17T07:57:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208010f500 msgr2=0x7f208019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208010f500 0x7f208019fcd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f207800b070 tx=0x7f2078004c80 comp rx=0 tx=0).stop 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 shutdown_connections 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f20640a3d50 0x7f20640a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208010f500 0x7f208019fcd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208006a760 0x7f208019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 --2- 172.21.15.46:0/2236668984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006b080 0x7f208019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 >> 172.21.15.46:0/2236668984 conn(0x7f2080078900 msgr2=0x7f20801108f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 shutdown_connections 2024-04-17T07:57:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:02.128+0000 7f2085800700 1 -- 172.21.15.46:0/2236668984 wait complete. 2024-04-17T07:57:03.201 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:02 smithi046 ceph-mon[32038]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:03.201 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:02 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2236668984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T07:57:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:02 smithi053 ceph-mon[36437]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:03.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:02 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2236668984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T07:57:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:02 smithi062 ceph-mon[36507]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:03.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:02 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2236668984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T07:57:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 -- 172.21.15.46:0/3320901515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848103b40 msgr2=0x7f4848103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 --2- 172.21.15.46:0/3320901515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848103b40 0x7f4848103f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f483c009990 tx=0x7f483c009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 -- 172.21.15.46:0/3320901515 shutdown_connections 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 --2- 172.21.15.46:0/3320901515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f484810cf70 0x7f484810f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 --2- 172.21.15.46:0/3320901515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848103b40 0x7f4848103f20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 --2- 172.21.15.46:0/3320901515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4848104460 0x7f484810c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 -- 172.21.15.46:0/3320901515 >> 172.21.15.46:0/3320901515 conn(0x7f48480fd370 msgr2=0x7f48480ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 -- 172.21.15.46:0/3320901515 shutdown_connections 2024-04-17T07:57:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.562+0000 7f484ea00700 1 -- 172.21.15.46:0/3320901515 wait complete. 2024-04-17T07:57:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.563+0000 7f484ea00700 1 Processor -- start 2024-04-17T07:57:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.563+0000 7f484ea00700 1 -- start start 2024-04-17T07:57:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4848103b40 0x7f48481017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848104460 0x7f4848101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f484810cf70 0x7f4848102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484ea00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4848112cd0 con 0x7f4848104460 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484ea00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4848112b50 con 0x7f484810cf70 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484ea00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4848112e50 con 0x7f4848103b40 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848104460 0x7f4848101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484e000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f484810cf70 0x7f4848102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848104460 0x7f4848101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53644/0 (socket says 172.21.15.46:53644) 2024-04-17T07:57:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484cc00700 1 -- 172.21.15.46:0/3620479274 learned_addr learned my addr 172.21.15.46:0/3620479274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.564+0000 7f484d600700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4848103b40 0x7f48481017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f484e000700 1 -- 172.21.15.46:0/3620479274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4848103b40 msgr2=0x7f48481017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f484e000700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4848103b40 0x7f48481017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f484e000700 1 -- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848104460 msgr2=0x7f4848101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f484e000700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848104460 0x7f4848101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f484e000700 1 -- 172.21.15.46:0/3620479274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48481a7670 con 0x7f484810cf70 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f484e000700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f484810cf70 0x7f4848102220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f484000efc0 tx=0x7f484000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f4846000700 1 -- 172.21.15.46:0/3620479274 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48400092e0 con 0x7f484810cf70 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f4846000700 1 -- 172.21.15.46:0/3620479274 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4840004bb0 con 0x7f484810cf70 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.565+0000 7f4846000700 1 -- 172.21.15.46:0/3620479274 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840004280 con 0x7f484810cf70 2024-04-17T07:57:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.566+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48481a7900 con 0x7f484810cf70 2024-04-17T07:57:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.566+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48481a7df0 con 0x7f484810cf70 2024-04-17T07:57:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.567+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4848069740 con 0x7f484810cf70 2024-04-17T07:57:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.569+0000 7f4846000700 1 -- 172.21.15.46:0/3620479274 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f484001b030 con 0x7f484810cf70 2024-04-17T07:57:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.570+0000 7f4846000700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48300a3cd0 0x7f48300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.570+0000 7f4846000700 1 -- 172.21.15.46:0/3620479274 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f48400d0760 con 0x7f484810cf70 2024-04-17T07:57:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.571+0000 7f484d600700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48300a3cd0 0x7f48300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.571+0000 7f484d600700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48300a3cd0 0x7f48300a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098e0 tx=0x7f483c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.573+0000 7f4846000700 1 -- 172.21.15.46:0/3620479274 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4840022020 con 0x7f484810cf70 2024-04-17T07:57:03.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.810+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f484802df90 con 0x7f484810cf70 2024-04-17T07:57:03.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.811+0000 7f4846000700 1 -- 172.21.15.46:0/3620479274 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4840010040 con 0x7f484810cf70 2024-04-17T07:57:03.811 INFO:teuthology.orchestra.run.smithi046.stdout:77309411414 2024-04-17T07:57:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48300a3cd0 msgr2=0x7f48300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48300a3cd0 0x7f48300a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f483c0098e0 tx=0x7f483c005960 comp rx=0 tx=0).stop 2024-04-17T07:57:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f484810cf70 msgr2=0x7f4848102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:03.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f484810cf70 0x7f4848102220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f484000efc0 tx=0x7f484000c3a0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 shutdown_connections 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48300a3cd0 0x7f48300a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f484810cf70 0x7f4848102220 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4848103b40 0x7f48481017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 --2- 172.21.15.46:0/3620479274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4848104460 0x7f4848101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 >> 172.21.15.46:0/3620479274 conn(0x7f48480fd370 msgr2=0x7f48480fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 shutdown_connections 2024-04-17T07:57:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.815+0000 7f484ea00700 1 -- 172.21.15.46:0/3620479274 wait complete. 2024-04-17T07:57:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.372+0000 7f0555200700 1 -- 172.21.15.46:0/382374774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550109fc0 msgr2=0x7f0550111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.372+0000 7f0555200700 1 --2- 172.21.15.46:0/382374774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550109fc0 0x7f0550111b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f054800b0d0 tx=0x7f054800b3e0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.373+0000 7f0555200700 1 -- 172.21.15.46:0/382374774 shutdown_connections 2024-04-17T07:57:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.373+0000 7f0555200700 1 --2- 172.21.15.46:0/382374774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550109fc0 0x7f0550111b90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.373+0000 7f0555200700 1 --2- 172.21.15.46:0/382374774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0550104ee0 0x7f05501052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.373+0000 7f0555200700 1 --2- 172.21.15.46:0/382374774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 0x7f05501098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.373+0000 7f0555200700 1 -- 172.21.15.46:0/382374774 >> 172.21.15.46:0/382374774 conn(0x7f0550100770 msgr2=0x7f0550102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.373+0000 7f0555200700 1 -- 172.21.15.46:0/382374774 shutdown_connections 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.373+0000 7f0555200700 1 -- 172.21.15.46:0/382374774 wait complete. 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 Processor -- start 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 -- start start 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550104ee0 0x7f055019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 0x7f055019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0550109fc0 0x7f055019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0550114fd0 con 0x7f0550104ee0 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0550114e50 con 0x7f0550109fc0 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.374+0000 7f0555200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0550115150 con 0x7f0550105890 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 0x7f055019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 0x7f055019b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:51708/0 (socket says 172.21.15.46:51708) 2024-04-17T07:57:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 -- 172.21.15.46:0/2588098799 learned_addr learned my addr 172.21.15.46:0/2588098799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f054de00700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550104ee0 0x7f055019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f054e800700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0550109fc0 0x7f055019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 -- 172.21.15.46:0/2588098799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0550109fc0 msgr2=0x7f055019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0550109fc0 0x7f055019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 -- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550104ee0 msgr2=0x7f055019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550104ee0 0x7f055019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.375+0000 7f0547e00700 1 -- 172.21.15.46:0/2588098799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05501a0410 con 0x7f0550105890 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.376+0000 7f0547e00700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 0x7f055019b940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9f0 tx=0x7f053800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.376+0000 7f0546a00700 1 -- 172.21.15.46:0/2588098799 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538004a30 con 0x7f0550105890 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.376+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05501a0700 con 0x7f0550105890 2024-04-17T07:57:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.376+0000 7f0546a00700 1 -- 172.21.15.46:0/2588098799 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053800cca0 con 0x7f0550105890 2024-04-17T07:57:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.376+0000 7f0546a00700 1 -- 172.21.15.46:0/2588098799 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380056c0 con 0x7f0550105890 2024-04-17T07:57:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.376+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05501a7f80 con 0x7f0550105890 2024-04-17T07:57:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.377+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0550069740 con 0x7f0550105890 2024-04-17T07:57:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.379+0000 7f0546a00700 1 -- 172.21.15.46:0/2588098799 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f053800c7c0 con 0x7f0550105890 2024-04-17T07:57:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.381+0000 7f0546a00700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f05340a3cd0 0x7f05340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.381+0000 7f0546a00700 1 -- 172.21.15.46:0/2588098799 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f05380d0a80 con 0x7f0550105890 2024-04-17T07:57:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.381+0000 7f054de00700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f05340a3cd0 0x7f05340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.382+0000 7f054de00700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f05340a3cd0 0x7f05340a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0540009c60 tx=0x7f0540009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.385+0000 7f0546a00700 1 -- 172.21.15.46:0/2588098799 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f05380d2e20 con 0x7f0550105890 2024-04-17T07:57:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.632+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f05501a8180 con 0x7f0550105890 2024-04-17T07:57:04.057 INFO:teuthology.orchestra.run.smithi046.stdout:107374182478 2024-04-17T07:57:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.633+0000 7f0546a00700 1 -- 172.21.15.46:0/2588098799 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f053809cf40 con 0x7f0550105890 2024-04-17T07:57:04.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f05340a3cd0 msgr2=0x7f05340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f05340a3cd0 0x7f05340a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0540009c60 tx=0x7f0540009260 comp rx=0 tx=0).stop 2024-04-17T07:57:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 msgr2=0x7f055019b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 0x7f055019b940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9f0 tx=0x7f053800ed00 comp rx=0 tx=0).stop 2024-04-17T07:57:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 shutdown_connections 2024-04-17T07:57:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f05340a3cd0 0x7f05340a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0550109fc0 0x7f055019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0550104ee0 0x7f055019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 --2- 172.21.15.46:0/2588098799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0550105890 0x7f055019b940 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.636+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 >> 172.21.15.46:0/2588098799 conn(0x7f0550100770 msgr2=0x7f05501011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.637+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 shutdown_connections 2024-04-17T07:57:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:03.637+0000 7f0555200700 1 -- 172.21.15.46:0/2588098799 wait complete. 2024-04-17T07:57:04.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:03 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2588098799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T07:57:04.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:03 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3620479274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T07:57:04.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:03 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2588098799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T07:57:04.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:03 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3620479274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T07:57:04.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:03 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2588098799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T07:57:04.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:03 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3620479274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T07:57:05.084 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:04 smithi046 ceph-mon[32038]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:04 smithi053 ceph-mon[36437]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:05.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:04 smithi062 ceph-mon[36507]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 -- 172.21.15.46:0/3466433507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103460 msgr2=0x7fb1a410f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 --2- 172.21.15.46:0/3466433507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103460 0x7fb1a410f8c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00b0a0 tx=0x7fb19c00b3b0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 -- 172.21.15.46:0/3466433507 shutdown_connections 2024-04-17T07:57:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 --2- 172.21.15.46:0/3466433507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103460 0x7fb1a410f8c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 --2- 172.21.15.46:0/3466433507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4108a60 0x7fb1a4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 --2- 172.21.15.46:0/3466433507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4102aa0 0x7fb1a4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 -- 172.21.15.46:0/3466433507 >> 172.21.15.46:0/3466433507 conn(0x7fb1a4079bb0 msgr2=0x7fb1a4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 -- 172.21.15.46:0/3466433507 shutdown_connections 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.904+0000 7fb1aa200700 1 -- 172.21.15.46:0/3466433507 wait complete. 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.905+0000 7fb1aa200700 1 Processor -- start 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1aa200700 1 -- start start 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1aa200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4102aa0 0x7fb1a419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1aa200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4103460 0x7fb1a419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1aa200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4108a60 0x7fb1a419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1aa200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4112dd0 con 0x7fb1a4108a60 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1aa200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4112c50 con 0x7fb1a4102aa0 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1aa200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4112f50 con 0x7fb1a4103460 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1a2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4103460 0x7fb1a419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1a2a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4103460 0x7fb1a419b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:51756/0 (socket says 172.21.15.46:51756) 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1a3400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4102aa0 0x7fb1a419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1a2a00700 1 -- 172.21.15.46:0/2938489377 learned_addr learned my addr 172.21.15.46:0/2938489377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.906+0000 7fb1a3e00700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4108a60 0x7fb1a419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.907+0000 7fb1a2a00700 1 -- 172.21.15.46:0/2938489377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4102aa0 msgr2=0x7fb1a419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.907+0000 7fb1a2a00700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4102aa0 0x7fb1a419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.907+0000 7fb1a2a00700 1 -- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4108a60 msgr2=0x7fb1a419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.907+0000 7fb1a2a00700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4108a60 0x7fb1a419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.907+0000 7fb1a2a00700 1 -- 172.21.15.46:0/2938489377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41a0410 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.907+0000 7fb1a2a00700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4103460 0x7fb1a419b940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9c0 tx=0x7fb19000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.907+0000 7fb19be00700 1 -- 172.21.15.46:0/2938489377 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19000cbd0 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.908+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41a0700 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.908+0000 7fb19be00700 1 -- 172.21.15.46:0/2938489377 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19000ee60 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.908+0000 7fb19be00700 1 -- 172.21.15.46:0/2938489377 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190005630 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.908+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41a7f80 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.909+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a4069740 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.911+0000 7fb19be00700 1 -- 172.21.15.46:0/2938489377 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb190004750 con 0x7fb1a4103460 2024-04-17T07:57:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.912+0000 7fb19be00700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3cd0 0x7fb1880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.912+0000 7fb19be00700 1 -- 172.21.15.46:0/2938489377 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb1900d0db0 con 0x7fb1a4103460 2024-04-17T07:57:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.913+0000 7fb1a3400700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3cd0 0x7fb1880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.913+0000 7fb1a3400700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3cd0 0x7fb1880a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb194009be0 tx=0x7fb194009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:04.915+0000 7fb19be00700 1 -- 172.21.15.46:0/2938489377 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb190053ff0 con 0x7fb1a4103460 2024-04-17T07:57:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.158+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb1a419c760 con 0x7fb1a4103460 2024-04-17T07:57:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:287762808876 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.159+0000 7fb19be00700 1 -- 172.21.15.46:0/2938489377 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb19009d270 con 0x7fb1a4103460 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3cd0 msgr2=0x7fb1880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3cd0 0x7fb1880a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb194009be0 tx=0x7fb194009260 comp rx=0 tx=0).stop 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4103460 msgr2=0x7fb1a419b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4103460 0x7fb1a419b940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9c0 tx=0x7fb19000ecd0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 shutdown_connections 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3cd0 0x7fb1880a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4108a60 0x7fb1a419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4102aa0 0x7fb1a419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 --2- 172.21.15.46:0/2938489377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4103460 0x7fb1a419b940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:05.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 >> 172.21.15.46:0/2938489377 conn(0x7fb1a4079bb0 msgr2=0x7fb1a410dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:05.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 shutdown_connections 2024-04-17T07:57:05.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:05.162+0000 7fb1aa200700 1 -- 172.21.15.46:0/2938489377 wait complete. 2024-04-17T07:57:06.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:05 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2938489377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T07:57:06.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:05 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2938489377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T07:57:06.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:05 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2938489377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T07:57:06.820 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182472 got 107374182478 for osd.3 2024-04-17T07:57:06.821 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 -- 172.21.15.46:0/2184508914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0107d50 msgr2=0x7f76c010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 --2- 172.21.15.46:0/2184508914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0107d50 0x7f76c010f900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f76b800b0a0 tx=0x7f76b800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:57:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 -- 172.21.15.46:0/2184508914 shutdown_connections 2024-04-17T07:57:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 --2- 172.21.15.46:0/2184508914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c0107d50 0x7f76c010f900 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 --2- 172.21.15.46:0/2184508914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c01014e0 0x7f76c01018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 --2- 172.21.15.46:0/2184508914 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c0101e90 0x7f76c0107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 -- 172.21.15.46:0/2184508914 >> 172.21.15.46:0/2184508914 conn(0x7f76c0076c00 msgr2=0x7f76c0077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 -- 172.21.15.46:0/2184508914 shutdown_connections 2024-04-17T07:57:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.937+0000 7f76bf200700 1 -- 172.21.15.46:0/2184508914 wait complete. 2024-04-17T07:57:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.938+0000 7f76bf200700 1 Processor -- start 2024-04-17T07:57:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.938+0000 7f76bf200700 1 -- start start 2024-04-17T07:57:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bf200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c01014e0 0x7f76c0102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bf200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c0101e90 0x7f76c01029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bf200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0107d50 0x7f76c0102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bf200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112d50 con 0x7f76c01014e0 2024-04-17T07:57:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bf200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112bd0 con 0x7f76c0107d50 2024-04-17T07:57:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bf200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112ed0 con 0x7f76c0101e90 2024-04-17T07:57:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c01014e0 0x7f76c0102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76be800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0107d50 0x7f76c0102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bde00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c01014e0 0x7f76c0102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53670/0 (socket says 172.21.15.46:53670) 2024-04-17T07:57:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76bde00700 1 -- 172.21.15.46:0/1050621137 learned_addr learned my addr 172.21.15.46:0/1050621137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.939+0000 7f76b7e00700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c0101e90 0x7f76c01029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76be800700 1 -- 172.21.15.46:0/1050621137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c0101e90 msgr2=0x7f76c01029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76be800700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c0101e90 0x7f76c01029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76be800700 1 -- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c01014e0 msgr2=0x7f76c0102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76be800700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c01014e0 0x7f76c0102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76be800700 1 -- 172.21.15.46:0/1050621137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c01a7630 con 0x7f76c0107d50 2024-04-17T07:57:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76bde00700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c01014e0 0x7f76c0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76b7e00700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c0101e90 0x7f76c01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76be800700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0107d50 0x7f76c0102ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76b800b0a0 tx=0x7f76b8004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76b6a00700 1 -- 172.21.15.46:0/1050621137 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8021070 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.940+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c01a78c0 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.941+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a7dc0 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.941+0000 7f76b6a00700 1 -- 172.21.15.46:0/1050621137 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b8007b80 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.941+0000 7f76b6a00700 1 -- 172.21.15.46:0/1050621137 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b801de00 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.942+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c0069740 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.945+0000 7f76b6a00700 1 -- 172.21.15.46:0/1050621137 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f76b8024070 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.946+0000 7f76b6a00700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76a40a3d50 0x7f76a40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.947+0000 7f76b6a00700 1 -- 172.21.15.46:0/1050621137 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f76b80d1390 con 0x7f76c0107d50 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.947+0000 7f76bde00700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76a40a3d50 0x7f76a40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:06.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.947+0000 7f76bde00700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76a40a3d50 0x7f76a40a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f76b0011550 tx=0x7f76b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:06.951+0000 7f76b6a00700 1 -- 172.21.15.46:0/1050621137 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f76b802b030 con 0x7f76c0107d50 2024-04-17T07:57:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.192+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f76c002df90 con 0x7f76c0107d50 2024-04-17T07:57:07.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:06 smithi046 ceph-mon[32038]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:07.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.193+0000 7f76b6a00700 1 -- 172.21.15.46:0/1050621137 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f76b809d8e0 con 0x7f76c0107d50 2024-04-17T07:57:07.193 INFO:teuthology.orchestra.run.smithi046.stdout:141733920842 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76a40a3d50 msgr2=0x7f76a40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76a40a3d50 0x7f76a40a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f76b0011550 tx=0x7f76b0009260 comp rx=0 tx=0).stop 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0107d50 msgr2=0x7f76c0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0107d50 0x7f76c0102ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76b800b0a0 tx=0x7f76b8004d00 comp rx=0 tx=0).stop 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 shutdown_connections 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76a40a3d50 0x7f76a40a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0107d50 0x7f76c0102ee0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c01014e0 0x7f76c0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 --2- 172.21.15.46:0/1050621137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76c0101e90 0x7f76c01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 >> 172.21.15.46:0/1050621137 conn(0x7f76c0076c00 msgr2=0x7f76c00ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:07.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.196+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 shutdown_connections 2024-04-17T07:57:07.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.197+0000 7f76bf200700 1 -- 172.21.15.46:0/1050621137 wait complete. 2024-04-17T07:57:07.204 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:06 smithi062 ceph-mon[36507]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:07.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:06 smithi053 ceph-mon[36437]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:07.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411414 for osd.2 2024-04-17T07:57:07.524 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 -- 172.21.15.46:0/1054334894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e01069a0 msgr2=0x7f27e0106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1054334894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e01069a0 0x7f27e0106d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009990 tx=0x7f27d0009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 -- 172.21.15.46:0/1054334894 shutdown_connections 2024-04-17T07:57:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1054334894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 0x7f27e010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1054334894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e01069a0 0x7f27e0106d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1054334894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e0101d30 0x7f27e01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 -- 172.21.15.46:0/1054334894 >> 172.21.15.46:0/1054334894 conn(0x7f27e0078900 msgr2=0x7f27e0078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 -- 172.21.15.46:0/1054334894 shutdown_connections 2024-04-17T07:57:07.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.500+0000 7f27e6a00700 1 -- 172.21.15.46:0/1054334894 wait complete. 2024-04-17T07:57:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.501+0000 7f27e6a00700 1 Processor -- start 2024-04-17T07:57:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27e6a00700 1 -- start start 2024-04-17T07:57:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27e6a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e0101d30 0x7f27e01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27e6a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 0x7f27e0103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27e6a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e01069a0 0x7f27e0104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27e6a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111c30 con 0x7f27e01069a0 2024-04-17T07:57:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27e6a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111ab0 con 0x7f27e01026f0 2024-04-17T07:57:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27e6a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111db0 con 0x7f27e0101d30 2024-04-17T07:57:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 0x7f27e0103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27d7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 0x7f27e0103b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42008/0 (socket says 172.21.15.46:42008) 2024-04-17T07:57:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.502+0000 7f27d7e00700 1 -- 172.21.15.46:0/1701072713 learned_addr learned my addr 172.21.15.46:0/1701072713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27dfe00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e0101d30 0x7f27e01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27e4a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e01069a0 0x7f27e0104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27d7e00700 1 -- 172.21.15.46:0/1701072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e0101d30 msgr2=0x7f27e01035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27d7e00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e0101d30 0x7f27e01035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27d7e00700 1 -- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e01069a0 msgr2=0x7f27e0104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27d7e00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e01069a0 0x7f27e0104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27d7e00700 1 -- 172.21.15.46:0/1701072713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e01a7710 con 0x7f27e01026f0 2024-04-17T07:57:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27e4a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e01069a0 0x7f27e0104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:57:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.503+0000 7f27d7e00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 0x7f27e0103b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9a0 tx=0x7f27c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.504+0000 7f27dd600700 1 -- 172.21.15.46:0/1701072713 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c800cbd0 con 0x7f27e01026f0 2024-04-17T07:57:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.504+0000 7f27dd600700 1 -- 172.21.15.46:0/1701072713 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c800cd70 con 0x7f27e01026f0 2024-04-17T07:57:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.504+0000 7f27dd600700 1 -- 172.21.15.46:0/1701072713 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8018750 con 0x7f27e01026f0 2024-04-17T07:57:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.504+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e01a7a00 con 0x7f27e01026f0 2024-04-17T07:57:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.504+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a7ef0 con 0x7f27e01026f0 2024-04-17T07:57:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.505+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e0103090 con 0x7f27e01026f0 2024-04-17T07:57:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.507+0000 7f27dd600700 1 -- 172.21.15.46:0/1701072713 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f27c8010430 con 0x7f27e01026f0 2024-04-17T07:57:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.508+0000 7f27dd600700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f27c40a3cd0 0x7f27c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.509+0000 7f27dd600700 1 -- 172.21.15.46:0/1701072713 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f27c80d0c00 con 0x7f27e01026f0 2024-04-17T07:57:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.509+0000 7f27dfe00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f27c40a3cd0 0x7f27c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.510+0000 7f27dfe00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f27c40a3cd0 0x7f27c40a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f27d00098e0 tx=0x7f27d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.511+0000 7f27dd600700 1 -- 172.21.15.46:0/1701072713 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f27c809d0c0 con 0x7f27e01026f0 2024-04-17T07:57:07.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.755+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f27e002df90 con 0x7f27e01026f0 2024-04-17T07:57:07.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.756+0000 7f27dd600700 1 -- 172.21.15.46:0/1701072713 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f27c809c9c0 con 0x7f27e01026f0 2024-04-17T07:57:07.755 INFO:teuthology.orchestra.run.smithi046.stdout:197568495678 2024-04-17T07:57:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f27c40a3cd0 msgr2=0x7f27c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f27c40a3cd0 0x7f27c40a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f27d00098e0 tx=0x7f27d0005960 comp rx=0 tx=0).stop 2024-04-17T07:57:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 msgr2=0x7f27e0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:07.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 0x7f27e0103b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9a0 tx=0x7f27c800ecb0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 shutdown_connections 2024-04-17T07:57:07.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f27c40a3cd0 0x7f27c40a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e01069a0 0x7f27e0104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27e0101d30 0x7f27e01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 --2- 172.21.15.46:0/1701072713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01026f0 0x7f27e0103b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:07.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.759+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 >> 172.21.15.46:0/1701072713 conn(0x7f27e0078900 msgr2=0x7f27e006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:07.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.760+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 shutdown_connections 2024-04-17T07:57:07.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:07.760+0000 7f27e6a00700 1 -- 172.21.15.46:0/1701072713 wait complete. 2024-04-17T07:57:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:07 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1050621137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T07:57:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:07 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1701072713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T07:57:08.238 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:07 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1050621137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T07:57:08.238 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:07 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1701072713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T07:57:08.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:07 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1050621137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T07:57:08.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:07 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1701072713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T07:57:09.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:08 smithi053 ceph-mon[36437]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:09.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:08 smithi046 ceph-mon[32038]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:09.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:08 smithi062 ceph-mon[36507]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:09.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808872 got 287762808876 for osd.9 2024-04-17T07:57:09.343 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:09.848 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332146 got 219043332151 for osd.7 2024-04-17T07:57:09.848 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:09.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920842 for osd.4 2024-04-17T07:57:09.966 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:10.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:09 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1946703307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T07:57:10.226 INFO:teuthology.orchestra.run.smithi046.stdout:163208757318 2024-04-17T07:57:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.306+0000 7f0dfb200700 1 -- 172.21.15.46:0/995881689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 msgr2=0x7f0dfc10f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.306+0000 7f0dfb200700 1 --2- 172.21.15.46:0/995881689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 0x7f0dfc10f850 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009990 tx=0x7f0df4009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.307+0000 7f0dfb200700 1 -- 172.21.15.46:0/995881689 shutdown_connections 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.307+0000 7f0dfb200700 1 --2- 172.21.15.46:0/995881689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 0x7f0dfc10f850 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.307+0000 7f0dfb200700 1 --2- 172.21.15.46:0/995881689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc108980 0x7f0dfc108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.307+0000 7f0dfb200700 1 --2- 172.21.15.46:0/995881689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dfc102980 0x7f0dfc102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.307+0000 7f0dfb200700 1 -- 172.21.15.46:0/995881689 >> 172.21.15.46:0/995881689 conn(0x7f0dfc0fe4d0 msgr2=0x7f0dfc1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.307+0000 7f0dfb200700 1 -- 172.21.15.46:0/995881689 shutdown_connections 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.307+0000 7f0dfb200700 1 -- 172.21.15.46:0/995881689 wait complete. 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.308+0000 7f0dfb200700 1 Processor -- start 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.308+0000 7f0dfb200700 1 -- start start 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0dfb200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc102980 0x7f0dfc1a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0dfb200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 0x7f0dfc1a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0dfb200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dfc108980 0x7f0dfc10e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0dfb200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112d10 con 0x7f0dfc103340 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0dfb200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112b90 con 0x7f0dfc108980 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0dfb200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112e90 con 0x7f0dfc102980 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc102980 0x7f0dfc1a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc102980 0x7f0dfc1a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40368/0 (socket says 172.21.15.46:40368) 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 -- 172.21.15.46:0/1946703307 learned_addr learned my addr 172.21.15.46:0/1946703307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df3e00700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 0x7f0dfc1a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0dfa800700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dfc108980 0x7f0dfc10e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 -- 172.21.15.46:0/1946703307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dfc108980 msgr2=0x7f0dfc10e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dfc108980 0x7f0dfc10e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 -- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 msgr2=0x7f0dfc1a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 0x7f0dfc1a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.309+0000 7f0df9e00700 1 -- 172.21.15.46:0/1946703307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc10f150 con 0x7f0dfc102980 2024-04-17T07:57:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.310+0000 7f0df9e00700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc102980 0x7f0dfc1a5380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00ef10 tx=0x7f0dec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.310+0000 7f0df2a00700 1 -- 172.21.15.46:0/1946703307 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00cad0 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.310+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc10f3e0 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.310+0000 7f0df2a00700 1 -- 172.21.15.46:0/1946703307 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dec019070 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.310+0000 7f0df2a00700 1 -- 172.21.15.46:0/1946703307 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0146e0 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.310+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc1ac1e0 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.312+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc069740 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.313+0000 7f0df2a00700 1 -- 172.21.15.46:0/1946703307 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0dec012050 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.315+0000 7f0df2a00700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0de00a3cd0 0x7f0de00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.315+0000 7f0df2a00700 1 -- 172.21.15.46:0/1946703307 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0dec0d0920 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.315+0000 7f0df3e00700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0de00a3cd0 0x7f0de00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.316+0000 7f0df3e00700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0de00a3cd0 0x7f0de00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc076d70 tx=0x7f0de4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.320+0000 7f0df2a00700 1 -- 172.21.15.46:0/1946703307 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0dec010040 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.563+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0dfc1ac440 con 0x7f0dfc102980 2024-04-17T07:57:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.563+0000 7f0df2a00700 1 -- 172.21.15.46:0/1946703307 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0dec09ce40 con 0x7f0dfc102980 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0de00a3cd0 msgr2=0x7f0de00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0de00a3cd0 0x7f0de00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc076d70 tx=0x7f0de4009260 comp rx=0 tx=0).stop 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc102980 msgr2=0x7f0dfc1a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc102980 0x7f0dfc1a5380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00ef10 tx=0x7f0dec00c3a0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 shutdown_connections 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0de00a3cd0 0x7f0de00a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dfc108980 0x7f0dfc10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dfc102980 0x7f0dfc1a5380 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 --2- 172.21.15.46:0/1946703307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dfc103340 0x7f0dfc1a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 >> 172.21.15.46:0/1946703307 conn(0x7f0dfc0fe4d0 msgr2=0x7f0dfc0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 shutdown_connections 2024-04-17T07:57:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:09.567+0000 7f0dfb200700 1 -- 172.21.15.46:0/1946703307 wait complete. 2024-04-17T07:57:10.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:09 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1946703307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T07:57:10.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:09 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1946703307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 -- 172.21.15.46:0/1564605140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43406b080 msgr2=0x7fe43410efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 --2- 172.21.15.46:0/1564605140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43406b080 0x7fe43410efc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe420009990 tx=0x7fe420009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 -- 172.21.15.46:0/1564605140 shutdown_connections 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 --2- 172.21.15.46:0/1564605140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43410f500 0x7fe4341118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 --2- 172.21.15.46:0/1564605140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe43406a760 0x7fe43406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 --2- 172.21.15.46:0/1564605140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43406b080 0x7fe43410efc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 -- 172.21.15.46:0/1564605140 >> 172.21.15.46:0/1564605140 conn(0x7fe434078900 msgr2=0x7fe434078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 -- 172.21.15.46:0/1564605140 shutdown_connections 2024-04-17T07:57:10.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.692+0000 7fe43c400700 1 -- 172.21.15.46:0/1564605140 wait complete. 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.693+0000 7fe43c400700 1 Processor -- start 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.693+0000 7fe43c400700 1 -- start start 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe43c400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43406a760 0x7fe43419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe43c400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe43406b080 0x7fe43419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe43c400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43410f500 0x7fe43419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe43c400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe434114fe0 con 0x7fe43406b080 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe43c400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe434114e60 con 0x7fe43406a760 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe43c400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe434115160 con 0x7fe43410f500 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe439000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe43406b080 0x7fe43419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe439a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43406a760 0x7fe43419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe439a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43406a760 0x7fe43419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40548/0 (socket says 172.21.15.46:40548) 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe439a00700 1 -- 172.21.15.46:0/3448958660 learned_addr learned my addr 172.21.15.46:0/3448958660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.694+0000 7fe43a400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43410f500 0x7fe43419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe439000700 1 -- 172.21.15.46:0/3448958660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43410f500 msgr2=0x7fe43419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe439000700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43410f500 0x7fe43419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe439000700 1 -- 172.21.15.46:0/3448958660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43406a760 msgr2=0x7fe43419b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe439000700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43406a760 0x7fe43419b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe439000700 1 -- 172.21.15.46:0/3448958660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4341a0500 con 0x7fe43406b080 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe43a400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43410f500 0x7fe43419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe439a00700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43406a760 0x7fe43419b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.695+0000 7fe439000700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe43406b080 0x7fe43419ba30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe420000c00 tx=0x7fe42000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.696+0000 7fe432000700 1 -- 172.21.15.46:0/3448958660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420026070 con 0x7fe43406b080 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.696+0000 7fe432000700 1 -- 172.21.15.46:0/3448958660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe420003d70 con 0x7fe43406b080 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.696+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4341a0790 con 0x7fe43406b080 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.696+0000 7fe432000700 1 -- 172.21.15.46:0/3448958660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42000fdc0 con 0x7fe43406b080 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.696+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4341a8070 con 0x7fe43406b080 2024-04-17T07:57:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.697+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe434101f70 con 0x7fe43406b080 2024-04-17T07:57:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.699+0000 7fe432000700 1 -- 172.21.15.46:0/3448958660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe420029070 con 0x7fe43406b080 2024-04-17T07:57:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.700+0000 7fe432000700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe41c0a3cd0 0x7fe41c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.701+0000 7fe432000700 1 -- 172.21.15.46:0/3448958660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe4200d1630 con 0x7fe43406b080 2024-04-17T07:57:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.701+0000 7fe439a00700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe41c0a3cd0 0x7fe41c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.701+0000 7fe439a00700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe41c0a3cd0 0x7fe41c0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe428011550 tx=0x7fe428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.705+0000 7fe432000700 1 -- 172.21.15.46:0/3448958660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe42005dff0 con 0x7fe43406b080 2024-04-17T07:57:10.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.940+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe43402df90 con 0x7fe43406b080 2024-04-17T07:57:10.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.941+0000 7fe432000700 1 -- 172.21.15.46:0/3448958660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe420013070 con 0x7fe43406b080 2024-04-17T07:57:10.941 INFO:teuthology.orchestra.run.smithi046.stdout:34359738464 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe41c0a3cd0 msgr2=0x7fe41c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe41c0a3cd0 0x7fe41c0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe428011550 tx=0x7fe428009260 comp rx=0 tx=0).stop 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe43406b080 msgr2=0x7fe43419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe43406b080 0x7fe43419ba30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe420000c00 tx=0x7fe42000bab0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 shutdown_connections 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe41c0a3cd0 0x7fe41c0a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe43410f500 0x7fe43419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe43406a760 0x7fe43419b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 --2- 172.21.15.46:0/3448958660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe43406b080 0x7fe43419ba30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 >> 172.21.15.46:0/3448958660 conn(0x7fe434078900 msgr2=0x7fe434110a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.944+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 shutdown_connections 2024-04-17T07:57:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:10.945+0000 7fe43c400700 1 -- 172.21.15.46:0/3448958660 wait complete. 2024-04-17T07:57:11.122 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757313 got 163208757318 for osd.5 2024-04-17T07:57:11.122 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:11.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:10 smithi062 ceph-mon[36507]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:11.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:10 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3448958660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T07:57:11.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:10 smithi046 ceph-mon[32038]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:11.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:10 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3448958660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T07:57:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:10 smithi053 ceph-mon[36437]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:10 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3448958660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T07:57:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:11.806 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738464 for osd.0 2024-04-17T07:57:11.806 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:12.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495674 got 197568495678 for osd.6 2024-04-17T07:57:12.235 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:13.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:12 smithi046 ceph-mon[32038]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:13.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:12 smithi062 ceph-mon[36507]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:13.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:12 smithi053 ceph-mon[36437]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:15.099 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:14 smithi046 ceph-mon[32038]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:15.099 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:14 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:15.099 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:14 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:14 smithi053 ceph-mon[36437]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:14 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:15.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:14 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:15.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:14 smithi062 ceph-mon[36507]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:15.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:14 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:15.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:14 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.631+0000 7f2f04400700 1 -- 172.21.15.46:0/1874752212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 msgr2=0x7f2efc108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.631+0000 7f2f04400700 1 --2- 172.21.15.46:0/1874752212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 0x7f2efc108ec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009990 tx=0x7f2ef0009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.632+0000 7f2f04400700 1 -- 172.21.15.46:0/1874752212 shutdown_connections 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.632+0000 7f2f04400700 1 --2- 172.21.15.46:0/1874752212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc1019e0 0x7f2efc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.632+0000 7f2f04400700 1 --2- 172.21.15.46:0/1874752212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 0x7f2efc108ec0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.632+0000 7f2f04400700 1 --2- 172.21.15.46:0/1874752212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2efc101040 0x7f2efc1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.632+0000 7f2f04400700 1 -- 172.21.15.46:0/1874752212 >> 172.21.15.46:0/1874752212 conn(0x7f2efc077fc0 msgr2=0x7f2efc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.632+0000 7f2f04400700 1 -- 172.21.15.46:0/1874752212 shutdown_connections 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.632+0000 7f2f04400700 1 -- 172.21.15.46:0/1874752212 wait complete. 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.633+0000 7f2f04400700 1 Processor -- start 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f04400700 1 -- start start 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f04400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2efc101040 0x7f2efc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f04400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc1019e0 0x7f2efc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f04400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 0x7f2efc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f04400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc112d80 con 0x7f2efc1019e0 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f04400700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc112c00 con 0x7f2efc101040 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f04400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc112f00 con 0x7f2efc108ae0 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f01a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2efc101040 0x7f2efc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.634+0000 7f2f01000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc1019e0 0x7f2efc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f01000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc1019e0 0x7f2efc19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40904/0 (socket says 172.21.15.46:40904) 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f02400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 0x7f2efc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f01a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2efc101040 0x7f2efc19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40572/0 (socket says 172.21.15.46:40572) 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f01000700 1 -- 172.21.15.46:0/1396893675 learned_addr learned my addr 172.21.15.46:0/1396893675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f02400700 1 -- 172.21.15.46:0/1396893675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2efc101040 msgr2=0x7f2efc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f02400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2efc101040 0x7f2efc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f02400700 1 -- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc1019e0 msgr2=0x7f2efc19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f02400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc1019e0 0x7f2efc19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f02400700 1 -- 172.21.15.46:0/1396893675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc1a0440 con 0x7f2efc108ae0 2024-04-17T07:57:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.635+0000 7f2f02400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 0x7f2efc19fd00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400efc0 tx=0x7f2ef400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.636+0000 7f2efa000700 1 -- 172.21.15.46:0/1396893675 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40092e0 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.636+0000 7f2efa000700 1 -- 172.21.15.46:0/1396893675 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef400ce20 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.636+0000 7f2efa000700 1 -- 172.21.15.46:0/1396893675 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4007c70 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.636+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc1a06d0 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.637+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc1a7fb0 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.638+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2efc104360 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.640+0000 7f2efa000700 1 -- 172.21.15.46:0/1396893675 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2ef401b020 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.641+0000 7f2efa000700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2ee40a3d50 0x7f2ee40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.642+0000 7f2efa000700 1 -- 172.21.15.46:0/1396893675 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2ef40d0e50 con 0x7f2efc108ae0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.642+0000 7f2f01a00700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2ee40a3d50 0x7f2ee40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.643+0000 7f2f01a00700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2ee40a3d50 0x7f2ee40a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006010 tx=0x7f2ef000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.644+0000 7f2efa000700 1 -- 172.21.15.46:0/1396893675 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2ef4010040 con 0x7f2efc108ae0 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 -- 172.21.15.46:0/553064177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a8107930 msgr2=0x7fc7a806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/553064177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a8107930 0x7fc7a806afc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7980098f0 tx=0x7fc798009c00 comp rx=0 tx=0).stop 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 -- 172.21.15.46:0/553064177 shutdown_connections 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/553064177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a81053d0 0x7fc7a8105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/553064177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a8107930 0x7fc7a806afc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/553064177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a806b500 0x7fc7a806b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 -- 172.21.15.46:0/553064177 >> 172.21.15.46:0/553064177 conn(0x7fc7a8078790 msgr2=0x7fc7a8078ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 -- 172.21.15.46:0/553064177 shutdown_connections 2024-04-17T07:57:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.735+0000 7fc7a7e00700 1 -- 172.21.15.46:0/553064177 wait complete. 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.736+0000 7fc7a7e00700 1 Processor -- start 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.736+0000 7fc7a7e00700 1 -- start start 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a806b500 0x7fc7a819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a81053d0 0x7fc7a819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a8107930 0x7fc7a81a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8119200 con 0x7fc7a8107930 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8119080 con 0x7fc7a81053d0 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8119380 con 0x7fc7a806b500 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a8107930 0x7fc7a81a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a6a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a806b500 0x7fc7a819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a8107930 0x7fc7a81a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40926/0 (socket says 172.21.15.46:40926) 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7400700 1 -- 172.21.15.46:0/782226515 learned_addr learned my addr 172.21.15.46:0/782226515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a6000700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a81053d0 0x7fc7a819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a6a00700 1 -- 172.21.15.46:0/782226515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a81053d0 msgr2=0x7fc7a819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a6a00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a81053d0 0x7fc7a819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a6a00700 1 -- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a8107930 msgr2=0x7fc7a81a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a6a00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a8107930 0x7fc7a81a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a6a00700 1 -- 172.21.15.46:0/782226515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a81a4660 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.737+0000 7fc7a7400700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a8107930 0x7fc7a81a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.738+0000 7fc7a6a00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a806b500 0x7fc7a819f650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc798000c00 tx=0x7fc79800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.738+0000 7fc79f400700 1 -- 172.21.15.46:0/782226515 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798005140 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.738+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a81a48f0 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.738+0000 7fc79f400700 1 -- 172.21.15.46:0/782226515 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79800b560 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.738+0000 7fc79f400700 1 -- 172.21.15.46:0/782226515 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798003a20 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.738+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a81ac1d0 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.739+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a8069740 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.741+0000 7fc79f400700 1 -- 172.21.15.46:0/782226515 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc79800ba10 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.743+0000 7fc79f400700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc78c0a3cd0 0x7fc78c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.743+0000 7fc79f400700 1 -- 172.21.15.46:0/782226515 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc7980d19f0 con 0x7fc7a806b500 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.743+0000 7fc7a6000700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc78c0a3cd0 0x7fc78c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.744+0000 7fc7a6000700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc78c0a3cd0 0x7fc78c0a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc7a806b250 tx=0x7fc794009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:15.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.745+0000 7fc79f400700 1 -- 172.21.15.46:0/782226515 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc798013030 con 0x7fc7a806b500 2024-04-17T07:57:15.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.880+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2efc1a8180 con 0x7f2efc108ae0 2024-04-17T07:57:15.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.880+0000 7f2efa000700 1 -- 172.21.15.46:0/1396893675 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ef409d310 con 0x7f2efc108ae0 2024-04-17T07:57:15.880 INFO:teuthology.orchestra.run.smithi046.stdout:55834574940 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2ee40a3d50 msgr2=0x7f2ee40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2ee40a3d50 0x7f2ee40a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006010 tx=0x7f2ef000b260 comp rx=0 tx=0).stop 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 msgr2=0x7f2efc19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 0x7f2efc19fd00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400efc0 tx=0x7f2ef400c3a0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 shutdown_connections 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2ee40a3d50 0x7f2ee40a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2efc108ae0 0x7f2efc19fd00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2efc101040 0x7f2efc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 --2- 172.21.15.46:0/1396893675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2efc1019e0 0x7f2efc19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.883+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 >> 172.21.15.46:0/1396893675 conn(0x7f2efc077fc0 msgr2=0x7f2efc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.884+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 shutdown_connections 2024-04-17T07:57:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.884+0000 7f2f04400700 1 -- 172.21.15.46:0/1396893675 wait complete. 2024-04-17T07:57:15.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.983+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc7a81a09b0 con 0x7fc7a806b500 2024-04-17T07:57:15.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.983+0000 7fc79f400700 1 -- 172.21.15.46:0/782226515 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc79809df40 con 0x7fc7a806b500 2024-04-17T07:57:15.983 INFO:teuthology.orchestra.run.smithi046.stdout:343597383717 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc78c0a3cd0 msgr2=0x7fc78c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc78c0a3cd0 0x7fc78c0a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc7a806b250 tx=0x7fc794009310 comp rx=0 tx=0).stop 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a806b500 msgr2=0x7fc7a819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a806b500 0x7fc7a819f650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc798000c00 tx=0x7fc79800fa00 comp rx=0 tx=0).stop 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 shutdown_connections 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc78c0a3cd0 0x7fc78c0a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7a8107930 0x7fc7a81a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7a806b500 0x7fc7a819f650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 --2- 172.21.15.46:0/782226515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7a81053d0 0x7fc7a819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.986+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 >> 172.21.15.46:0/782226515 conn(0x7fc7a8078790 msgr2=0x7fc7a8100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.987+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 shutdown_connections 2024-04-17T07:57:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:15.987+0000 7fc7a7e00700 1 -- 172.21.15.46:0/782226515 wait complete. 2024-04-17T07:57:16.185 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:15 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:57:16.185 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:15 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:57:16.185 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:15 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:16.185 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:15 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1396893675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T07:57:16.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:15 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:57:16.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:15 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:57:16.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:15 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:16.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:15 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1396893675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T07:57:16.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:15 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:57:16.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:15 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:57:16.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:15 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:57:16.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:15 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1396893675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T07:57:17.069 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383710 got 343597383717 for osd.11 2024-04-17T07:57:17.069 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:17.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:16 smithi046 ceph-mon[32038]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:17.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:16 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/782226515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T07:57:17.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:16 smithi053 ceph-mon[36437]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:17.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:16 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/782226515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T07:57:17.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:16 smithi062 ceph-mon[36507]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:17.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:16 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/782226515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T07:57:17.381 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574934 got 55834574940 for osd.1 2024-04-17T07:57:17.381 DEBUG:teuthology.parallel:result is None 2024-04-17T07:57:17.381 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-17T07:57:17.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T07:57:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:19.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:18 smithi046 ceph-mon[32038]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:18 smithi053 ceph-mon[36437]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:19.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:18 smithi062 ceph-mon[36507]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 -- 172.21.15.46:0/1833338805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4109fd0 msgr2=0x7f0bc4111ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 --2- 172.21.15.46:0/1833338805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4109fd0 0x7f0bc4111ba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00b0d0 tx=0x7f0bbc00b3e0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 -- 172.21.15.46:0/1833338805 shutdown_connections 2024-04-17T07:57:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 --2- 172.21.15.46:0/1833338805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc4109fd0 0x7f0bc4111ba0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 --2- 172.21.15.46:0/1833338805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc4104ef0 0x7f0bc41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 --2- 172.21.15.46:0/1833338805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc41058a0 0x7f0bc41098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 -- 172.21.15.46:0/1833338805 >> 172.21.15.46:0/1833338805 conn(0x7f0bc4100780 msgr2=0x7f0bc4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:20.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 -- 172.21.15.46:0/1833338805 shutdown_connections 2024-04-17T07:57:20.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.097+0000 7f0bcc600700 1 -- 172.21.15.46:0/1833338805 wait complete. 2024-04-17T07:57:20.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.098+0000 7f0bcc600700 1 Processor -- start 2024-04-17T07:57:20.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bcc600700 1 -- start start 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bcc600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc4104ef0 0x7f0bc419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bcc600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc41058a0 0x7f0bc419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bcc600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc4109fd0 0x7f0bc419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bcc600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4115020 con 0x7f0bc4109fd0 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bcc600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc4114ea0 con 0x7f0bc4104ef0 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bcc600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc41151a0 con 0x7f0bc41058a0 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bc9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc4104ef0 0x7f0bc419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bc9c00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc4104ef0 0x7f0bc419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56930/0 (socket says 172.21.15.46:56930) 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.099+0000 7f0bc9c00700 1 -- 172.21.15.46:0/4152098545 learned_addr learned my addr 172.21.15.46:0/4152098545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bca600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc4109fd0 0x7f0bc419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9200700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc41058a0 0x7f0bc419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9200700 1 -- 172.21.15.46:0/4152098545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc4104ef0 msgr2=0x7f0bc419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:20.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9200700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc4104ef0 0x7f0bc419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9200700 1 -- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc4109fd0 msgr2=0x7f0bc419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:20.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9200700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc4109fd0 0x7f0bc419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9200700 1 -- 172.21.15.46:0/4152098545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc41a0450 con 0x7f0bc41058a0 2024-04-17T07:57:20.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9c00700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc4104ef0 0x7f0bc419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:20.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bca600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc4109fd0 0x7f0bc419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:20.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.100+0000 7f0bc9200700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc41058a0 0x7f0bc419b9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000e9c0 tx=0x7f0bb000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:20.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.101+0000 7f0bc2000700 1 -- 172.21.15.46:0/4152098545 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0004220 con 0x7f0bc41058a0 2024-04-17T07:57:20.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.101+0000 7f0bc2000700 1 -- 172.21.15.46:0/4152098545 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0004d10 con 0x7f0bc41058a0 2024-04-17T07:57:20.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.101+0000 7f0bc2000700 1 -- 172.21.15.46:0/4152098545 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0005020 con 0x7f0bc41058a0 2024-04-17T07:57:20.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.101+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc41a0740 con 0x7f0bc41058a0 2024-04-17T07:57:20.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.101+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a7fc0 con 0x7f0bc41058a0 2024-04-17T07:57:20.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.102+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc4069740 con 0x7f0bc41058a0 2024-04-17T07:57:20.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.104+0000 7f0bc2000700 1 -- 172.21.15.46:0/4152098545 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bb0004750 con 0x7f0bc41058a0 2024-04-17T07:57:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.105+0000 7f0bc2000700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0bac0a3cd0 0x7f0bac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.106+0000 7f0bc2000700 1 -- 172.21.15.46:0/4152098545 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f0bb00d0820 con 0x7f0bc41058a0 2024-04-17T07:57:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.106+0000 7f0bc9c00700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0bac0a3cd0 0x7f0bac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.106+0000 7f0bc9c00700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0bac0a3cd0 0x7f0bac0a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8011630 tx=0x7f0bb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:20.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.110+0000 7f0bc2000700 1 -- 172.21.15.46:0/4152098545 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0bb009cce0 con 0x7f0bc41058a0 2024-04-17T07:57:20.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.337+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bc4062e90 con 0x7f0bac0a3cd0 2024-04-17T07:57:20.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.368+0000 7f0bc2000700 1 -- 172.21.15.46:0/4152098545 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340358 (secure 0 0 0) 0x7f0bc4062e90 con 0x7f0bac0a3cd0 2024-04-17T07:57:20.375 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0bac0a3cd0 msgr2=0x7f0bac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0bac0a3cd0 0x7f0bac0a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8011630 tx=0x7f0bb8009260 comp rx=0 tx=0).stop 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc41058a0 msgr2=0x7f0bc419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc41058a0 0x7f0bc419b9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000e9c0 tx=0x7f0bb000ecd0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 shutdown_connections 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0bac0a3cd0 0x7f0bac0a6190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc4109fd0 0x7f0bc419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc4104ef0 0x7f0bc419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 --2- 172.21.15.46:0/4152098545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bc41058a0 0x7f0bc419b9d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 >> 172.21.15.46:0/4152098545 conn(0x7f0bc4100780 msgr2=0x7f0bc4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 shutdown_connections 2024-04-17T07:57:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:20.374+0000 7f0bcc600700 1 -- 172.21.15.46:0/4152098545 wait complete. 2024-04-17T07:57:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:57:21.165 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-04-17T07:57:19.554171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657619,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":649,"num_read_kb":443,"num_write":348,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375788,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566036,"statfs":{"total":1151940427776,"available":1151555620864,"internally_reserved":0,"allocated":14434304,"data_stored":7958567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006452"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819593+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:56:22.819593+0000","last_peered":"2024-04-17T07:56:22.819593+0000","last_clean":"2024-04-17T07:56:22.819593+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:56:22.819593+0000","last_undegraded":"2024-04-17T07:56:22.819593+0000","last_fullsized":"2024-04-17T07:56:22.819593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817631+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:56:22.817631+0000","last_peered":"2024-04-17T07:56:22.817631+0000","last_clean":"2024-04-17T07:56:22.817631+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:56:22.817631+0000","last_undegraded":"2024-04-17T07:56:22.817631+0000","last_fullsized":"2024-04-17T07:56:22.817631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815576+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:56:22.815576+0000","last_peered":"2024-04-17T07:56:22.815576+0000","last_clean":"2024-04-17T07:56:22.815576+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:56:22.815576+0000","last_undegraded":"2024-04-17T07:56:22.815576+0000","last_fullsized":"2024-04-17T07:56:22.815576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812907+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:56:22.812907+0000","last_peered":"2024-04-17T07:56:22.812907+0000","last_clean":"2024-04-17T07:56:22.812907+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:56:22.812907+0000","last_undegraded":"2024-04-17T07:56:22.812907+0000","last_fullsized":"2024-04-17T07:56:22.812907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814829+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:56:22.814829+0000","last_peered":"2024-04-17T07:56:22.814829+0000","last_clean":"2024-04-17T07:56:22.814829+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:56:22.814829+0000","last_undegraded":"2024-04-17T07:56:22.814829+0000","last_fullsized":"2024-04-17T07:56:22.814829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815736+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:56:22.815736+0000","last_peered":"2024-04-17T07:56:22.815736+0000","last_clean":"2024-04-17T07:56:22.815736+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:56:22.815736+0000","last_undegraded":"2024-04-17T07:56:22.815736+0000","last_fullsized":"2024-04-17T07:56:22.815736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814856+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:56:22.814856+0000","last_peered":"2024-04-17T07:56:22.814856+0000","last_clean":"2024-04-17T07:56:22.814856+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:56:22.814856+0000","last_undegraded":"2024-04-17T07:56:22.814856+0000","last_fullsized":"2024-04-17T07:56:22.814856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.830117+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:56:22.830117+0000","last_peered":"2024-04-17T07:56:22.830117+0000","last_clean":"2024-04-17T07:56:22.830117+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:56:22.830117+0000","last_undegraded":"2024-04-17T07:56:22.830117+0000","last_fullsized":"2024-04-17T07:56:22.830117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582310+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:56:23.582310+0000","last_peered":"2024-04-17T07:56:23.582310+0000","last_clean":"2024-04-17T07:56:23.582310+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:56:23.582310+0000","last_undegraded":"2024-04-17T07:56:23.582310+0000","last_fullsized":"2024-04-17T07:56:23.582310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816318+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:56:22.816318+0000","last_peered":"2024-04-17T07:56:22.816318+0000","last_clean":"2024-04-17T07:56:22.816318+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:56:22.816318+0000","last_undegraded":"2024-04-17T07:56:22.816318+0000","last_fullsized":"2024-04-17T07:56:22.816318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816276+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:56:22.816276+0000","last_peered":"2024-04-17T07:56:22.816276+0000","last_clean":"2024-04-17T07:56:22.816276+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:56:22.816276+0000","last_undegraded":"2024-04-17T07:56:22.816276+0000","last_fullsized":"2024-04-17T07:56:22.816276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812867+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:56:22.812867+0000","last_peered":"2024-04-17T07:56:22.812867+0000","last_clean":"2024-04-17T07:56:22.812867+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:56:22.812867+0000","last_undegraded":"2024-04-17T07:56:22.812867+0000","last_fullsized":"2024-04-17T07:56:22.812867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.826013+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:56:22.826013+0000","last_peered":"2024-04-17T07:56:22.826013+0000","last_clean":"2024-04-17T07:56:22.826013+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:56:22.826013+0000","last_undegraded":"2024-04-17T07:56:22.826013+0000","last_fullsized":"2024-04-17T07:56:22.826013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814974+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:56:22.814974+0000","last_peered":"2024-04-17T07:56:22.814974+0000","last_clean":"2024-04-17T07:56:22.814974+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:56:22.814974+0000","last_undegraded":"2024-04-17T07:56:22.814974+0000","last_fullsized":"2024-04-17T07:56:22.814974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816473+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:56:22.816473+0000","last_peered":"2024-04-17T07:56:22.816473+0000","last_clean":"2024-04-17T07:56:22.816473+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:56:22.816473+0000","last_undegraded":"2024-04-17T07:56:22.816473+0000","last_fullsized":"2024-04-17T07:56:22.816473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.581774+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:56:23.581774+0000","last_peered":"2024-04-17T07:56:23.581774+0000","last_clean":"2024-04-17T07:56:23.581774+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:56:23.581774+0000","last_undegraded":"2024-04-17T07:56:23.581774+0000","last_fullsized":"2024-04-17T07:56:23.581774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819510+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:56:22.819510+0000","last_peered":"2024-04-17T07:56:22.819510+0000","last_clean":"2024-04-17T07:56:22.819510+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:56:22.819510+0000","last_undegraded":"2024-04-17T07:56:22.819510+0000","last_fullsized":"2024-04-17T07:56:22.819510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829471+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:56:22.829471+0000","last_peered":"2024-04-17T07:56:22.829471+0000","last_clean":"2024-04-17T07:56:22.829471+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:56:22.829471+0000","last_undegraded":"2024-04-17T07:56:22.829471+0000","last_fullsized":"2024-04-17T07:56:22.829471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813374+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:56:22.813374+0000","last_peered":"2024-04-17T07:56:22.813374+0000","last_clean":"2024-04-17T07:56:22.813374+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:56:22.813374+0000","last_undegraded":"2024-04-17T07:56:22.813374+0000","last_fullsized":"2024-04-17T07:56:22.813374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812665+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:56:22.812665+0000","last_peered":"2024-04-17T07:56:22.812665+0000","last_clean":"2024-04-17T07:56:22.812665+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:56:22.812665+0000","last_undegraded":"2024-04-17T07:56:22.812665+0000","last_fullsized":"2024-04-17T07:56:22.812665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.835085+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:56:22.835085+0000","last_peered":"2024-04-17T07:56:22.835085+0000","last_clean":"2024-04-17T07:56:22.835085+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:56:22.835085+0000","last_undegraded":"2024-04-17T07:56:22.835085+0000","last_fullsized":"2024-04-17T07:56:22.835085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.834799+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:56:22.834799+0000","last_peered":"2024-04-17T07:56:22.834799+0000","last_clean":"2024-04-17T07:56:22.834799+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:56:22.834799+0000","last_undegraded":"2024-04-17T07:56:22.834799+0000","last_fullsized":"2024-04-17T07:56:22.834799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812293+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:56:22.812293+0000","last_peered":"2024-04-17T07:56:22.812293+0000","last_clean":"2024-04-17T07:56:22.812293+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:56:22.812293+0000","last_undegraded":"2024-04-17T07:56:22.812293+0000","last_fullsized":"2024-04-17T07:56:22.812293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.835880+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:56:22.835880+0000","last_peered":"2024-04-17T07:56:22.835880+0000","last_clean":"2024-04-17T07:56:22.835880+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:56:22.835880+0000","last_undegraded":"2024-04-17T07:56:22.835880+0000","last_fullsized":"2024-04-17T07:56:22.835880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816959+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:56:22.816959+0000","last_peered":"2024-04-17T07:56:22.816959+0000","last_clean":"2024-04-17T07:56:22.816959+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:56:22.816959+0000","last_undegraded":"2024-04-17T07:56:22.816959+0000","last_fullsized":"2024-04-17T07:56:22.816959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812385+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:56:22.812385+0000","last_peered":"2024-04-17T07:56:22.812385+0000","last_clean":"2024-04-17T07:56:22.812385+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:56:22.812385+0000","last_undegraded":"2024-04-17T07:56:22.812385+0000","last_fullsized":"2024-04-17T07:56:22.812385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814760+0000","last_change":"2024-04-17T07:55:25.509157+0000","last_active":"2024-04-17T07:56:22.814760+0000","last_peered":"2024-04-17T07:56:22.814760+0000","last_clean":"2024-04-17T07:56:22.814760+0000","last_became_active":"2024-04-17T07:55:25.508793+0000","last_became_peered":"2024-04-17T07:55:25.508793+0000","last_unstale":"2024-04-17T07:56:22.814760+0000","last_undegraded":"2024-04-17T07:56:22.814760+0000","last_fullsized":"2024-04-17T07:56:22.814760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:55:29.596338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582292+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:56:23.582292+0000","last_peered":"2024-04-17T07:56:23.582292+0000","last_clean":"2024-04-17T07:56:23.582292+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:56:23.582292+0000","last_undegraded":"2024-04-17T07:56:23.582292+0000","last_fullsized":"2024-04-17T07:56:23.582292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812345+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:56:22.812345+0000","last_peered":"2024-04-17T07:56:22.812345+0000","last_clean":"2024-04-17T07:56:22.812345+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:56:22.812345+0000","last_undegraded":"2024-04-17T07:56:22.812345+0000","last_fullsized":"2024-04-17T07:56:22.812345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814627+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:56:22.814627+0000","last_peered":"2024-04-17T07:56:22.814627+0000","last_clean":"2024-04-17T07:56:22.814627+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:56:22.814627+0000","last_undegraded":"2024-04-17T07:56:22.814627+0000","last_fullsized":"2024-04-17T07:56:22.814627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818056+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:56:22.818056+0000","last_peered":"2024-04-17T07:56:22.818056+0000","last_clean":"2024-04-17T07:56:22.818056+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:56:22.818056+0000","last_undegraded":"2024-04-17T07:56:22.818056+0000","last_fullsized":"2024-04-17T07:56:22.818056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813003+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:56:22.813003+0000","last_peered":"2024-04-17T07:56:22.813003+0000","last_clean":"2024-04-17T07:56:22.813003+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:56:22.813003+0000","last_undegraded":"2024-04-17T07:56:22.813003+0000","last_fullsized":"2024-04-17T07:56:22.813003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:14.661468+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:57:14.661468+0000","last_peered":"2024-04-17T07:57:14.661468+0000","last_clean":"2024-04-17T07:57:14.661468+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:57:14.661468+0000","last_undegraded":"2024-04-17T07:57:14.661468+0000","last_fullsized":"2024-04-17T07:57:14.661468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815118+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:56:22.815118+0000","last_peered":"2024-04-17T07:56:22.815118+0000","last_clean":"2024-04-17T07:56:22.815118+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:56:22.815118+0000","last_undegraded":"2024-04-17T07:56:22.815118+0000","last_fullsized":"2024-04-17T07:56:22.815118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812838+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:56:22.812838+0000","last_peered":"2024-04-17T07:56:22.812838+0000","last_clean":"2024-04-17T07:56:22.812838+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:56:22.812838+0000","last_undegraded":"2024-04-17T07:56:22.812838+0000","last_fullsized":"2024-04-17T07:56:22.812838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815586+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:56:22.815586+0000","last_peered":"2024-04-17T07:56:22.815586+0000","last_clean":"2024-04-17T07:56:22.815586+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:56:22.815586+0000","last_undegraded":"2024-04-17T07:56:22.815586+0000","last_fullsized":"2024-04-17T07:56:22.815586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818071+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:56:22.818071+0000","last_peered":"2024-04-17T07:56:22.818071+0000","last_clean":"2024-04-17T07:56:22.818071+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:56:22.818071+0000","last_undegraded":"2024-04-17T07:56:22.818071+0000","last_fullsized":"2024-04-17T07:56:22.818071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819451+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:56:22.819451+0000","last_peered":"2024-04-17T07:56:22.819451+0000","last_clean":"2024-04-17T07:56:22.819451+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:56:22.819451+0000","last_undegraded":"2024-04-17T07:56:22.819451+0000","last_fullsized":"2024-04-17T07:56:22.819451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.824775+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:56:22.824775+0000","last_peered":"2024-04-17T07:56:22.824775+0000","last_clean":"2024-04-17T07:56:22.824775+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:56:22.824775+0000","last_undegraded":"2024-04-17T07:56:22.824775+0000","last_fullsized":"2024-04-17T07:56:22.824775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829939+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:56:22.829939+0000","last_peered":"2024-04-17T07:56:22.829939+0000","last_clean":"2024-04-17T07:56:22.829939+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:56:22.829939+0000","last_undegraded":"2024-04-17T07:56:22.829939+0000","last_fullsized":"2024-04-17T07:56:22.829939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817465+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:56:22.817465+0000","last_peered":"2024-04-17T07:56:22.817465+0000","last_clean":"2024-04-17T07:56:22.817465+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:56:22.817465+0000","last_undegraded":"2024-04-17T07:56:22.817465+0000","last_fullsized":"2024-04-17T07:56:22.817465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819160+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:56:22.819160+0000","last_peered":"2024-04-17T07:56:22.819160+0000","last_clean":"2024-04-17T07:56:22.819160+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:56:22.819160+0000","last_undegraded":"2024-04-17T07:56:22.819160+0000","last_fullsized":"2024-04-17T07:56:22.819160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.827202+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:56:22.827202+0000","last_peered":"2024-04-17T07:56:22.827202+0000","last_clean":"2024-04-17T07:56:22.827202+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:56:22.827202+0000","last_undegraded":"2024-04-17T07:56:22.827202+0000","last_fullsized":"2024-04-17T07:56:22.827202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.831569+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:56:22.831569+0000","last_peered":"2024-04-17T07:56:22.831569+0000","last_clean":"2024-04-17T07:56:22.831569+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:56:22.831569+0000","last_undegraded":"2024-04-17T07:56:22.831569+0000","last_fullsized":"2024-04-17T07:56:22.831569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:15.662525+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:57:15.662525+0000","last_peered":"2024-04-17T07:57:15.662525+0000","last_clean":"2024-04-17T07:57:15.662525+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:57:15.662525+0000","last_undegraded":"2024-04-17T07:57:15.662525+0000","last_fullsized":"2024-04-17T07:57:15.662525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582435+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:56:23.582435+0000","last_peered":"2024-04-17T07:56:23.582435+0000","last_clean":"2024-04-17T07:56:23.582435+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:56:23.582435+0000","last_undegraded":"2024-04-17T07:56:23.582435+0000","last_fullsized":"2024-04-17T07:56:23.582435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.828709+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:56:22.828709+0000","last_peered":"2024-04-17T07:56:22.828709+0000","last_clean":"2024-04-17T07:56:22.828709+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:56:22.828709+0000","last_undegraded":"2024-04-17T07:56:22.828709+0000","last_fullsized":"2024-04-17T07:56:22.828709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815605+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:56:22.815605+0000","last_peered":"2024-04-17T07:56:22.815605+0000","last_clean":"2024-04-17T07:56:22.815605+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:56:22.815605+0000","last_undegraded":"2024-04-17T07:56:22.815605+0000","last_fullsized":"2024-04-17T07:56:22.815605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816876+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:56:22.816876+0000","last_peered":"2024-04-17T07:56:22.816876+0000","last_clean":"2024-04-17T07:56:22.816876+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:56:22.816876+0000","last_undegraded":"2024-04-17T07:56:22.816876+0000","last_fullsized":"2024-04-17T07:56:22.816876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813057+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:56:22.813057+0000","last_peered":"2024-04-17T07:56:22.813057+0000","last_clean":"2024-04-17T07:56:22.813057+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:56:22.813057+0000","last_undegraded":"2024-04-17T07:56:22.813057+0000","last_fullsized":"2024-04-17T07:56:22.813057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817917+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:56:22.817917+0000","last_peered":"2024-04-17T07:56:22.817917+0000","last_clean":"2024-04-17T07:56:22.817917+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:56:22.817917+0000","last_undegraded":"2024-04-17T07:56:22.817917+0000","last_fullsized":"2024-04-17T07:56:22.817917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582424+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:56:23.582424+0000","last_peered":"2024-04-17T07:56:23.582424+0000","last_clean":"2024-04-17T07:56:23.582424+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:56:23.582424+0000","last_undegraded":"2024-04-17T07:56:23.582424+0000","last_fullsized":"2024-04-17T07:56:23.582424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.830785+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:56:22.830785+0000","last_peered":"2024-04-17T07:56:22.830785+0000","last_clean":"2024-04-17T07:56:22.830785+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:56:22.830785+0000","last_undegraded":"2024-04-17T07:56:22.830785+0000","last_fullsized":"2024-04-17T07:56:22.830785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":233,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:16.664370+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:57:16.664370+0000","last_peered":"2024-04-17T07:57:16.664370+0000","last_clean":"2024-04-17T07:57:16.664370+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:57:16.664370+0000","last_undegraded":"2024-04-17T07:57:16.664370+0000","last_fullsized":"2024-04-17T07:57:16.664370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62625,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":174,"num_read_kb":149,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815354+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:56:22.815354+0000","last_peered":"2024-04-17T07:56:22.815354+0000","last_clean":"2024-04-17T07:56:22.815354+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:56:22.815354+0000","last_undegraded":"2024-04-17T07:56:22.815354+0000","last_fullsized":"2024-04-17T07:56:22.815354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818273+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:56:22.818273+0000","last_peered":"2024-04-17T07:56:22.818273+0000","last_clean":"2024-04-17T07:56:22.818273+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:56:22.818273+0000","last_undegraded":"2024-04-17T07:56:22.818273+0000","last_fullsized":"2024-04-17T07:56:22.818273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812853+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:56:22.812853+0000","last_peered":"2024-04-17T07:56:22.812853+0000","last_clean":"2024-04-17T07:56:22.812853+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:56:22.812853+0000","last_undegraded":"2024-04-17T07:56:22.812853+0000","last_fullsized":"2024-04-17T07:56:22.812853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.825904+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:56:22.825904+0000","last_peered":"2024-04-17T07:56:22.825904+0000","last_clean":"2024-04-17T07:56:22.825904+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:56:22.825904+0000","last_undegraded":"2024-04-17T07:56:22.825904+0000","last_fullsized":"2024-04-17T07:56:22.825904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819176+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:56:22.819176+0000","last_peered":"2024-04-17T07:56:22.819176+0000","last_clean":"2024-04-17T07:56:22.819176+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:56:22.819176+0000","last_undegraded":"2024-04-17T07:56:22.819176+0000","last_fullsized":"2024-04-17T07:56:22.819176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817949+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:56:22.817949+0000","last_peered":"2024-04-17T07:56:22.817949+0000","last_clean":"2024-04-17T07:56:22.817949+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:56:22.817949+0000","last_undegraded":"2024-04-17T07:56:22.817949+0000","last_fullsized":"2024-04-17T07:56:22.817949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.834205+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:56:22.834205+0000","last_peered":"2024-04-17T07:56:22.834205+0000","last_clean":"2024-04-17T07:56:22.834205+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:56:22.834205+0000","last_undegraded":"2024-04-17T07:56:22.834205+0000","last_fullsized":"2024-04-17T07:56:22.834205+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829626+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:56:22.829626+0000","last_peered":"2024-04-17T07:56:22.829626+0000","last_clean":"2024-04-17T07:56:22.829626+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:56:22.829626+0000","last_undegraded":"2024-04-17T07:56:22.829626+0000","last_fullsized":"2024-04-17T07:56:22.829626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:14.705116+0000","last_change":"2024-04-17T07:55:25.507054+0000","last_active":"2024-04-17T07:57:14.705116+0000","last_peered":"2024-04-17T07:57:14.705116+0000","last_clean":"2024-04-17T07:57:14.705116+0000","last_became_active":"2024-04-17T07:55:25.506709+0000","last_became_peered":"2024-04-17T07:55:25.506709+0000","last_unstale":"2024-04-17T07:57:14.705116+0000","last_undegraded":"2024-04-17T07:57:14.705116+0000","last_fullsized":"2024-04-17T07:57:14.705116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:53:29.194193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814698+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:56:22.814698+0000","last_peered":"2024-04-17T07:56:22.814698+0000","last_clean":"2024-04-17T07:56:22.814698+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:56:22.814698+0000","last_undegraded":"2024-04-17T07:56:22.814698+0000","last_fullsized":"2024-04-17T07:56:22.814698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.837796+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:56:22.837796+0000","last_peered":"2024-04-17T07:56:22.837796+0000","last_clean":"2024-04-17T07:56:22.837796+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:56:22.837796+0000","last_undegraded":"2024-04-17T07:56:22.837796+0000","last_fullsized":"2024-04-17T07:56:22.837796+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814480+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:56:22.814480+0000","last_peered":"2024-04-17T07:56:22.814480+0000","last_clean":"2024-04-17T07:56:22.814480+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:56:22.814480+0000","last_undegraded":"2024-04-17T07:56:22.814480+0000","last_fullsized":"2024-04-17T07:56:22.814480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817885+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:56:22.817885+0000","last_peered":"2024-04-17T07:56:22.817885+0000","last_clean":"2024-04-17T07:56:22.817885+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:56:22.817885+0000","last_undegraded":"2024-04-17T07:56:22.817885+0000","last_fullsized":"2024-04-17T07:56:22.817885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817725+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:56:22.817725+0000","last_peered":"2024-04-17T07:56:22.817725+0000","last_clean":"2024-04-17T07:56:22.817725+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:56:22.817725+0000","last_undegraded":"2024-04-17T07:56:22.817725+0000","last_fullsized":"2024-04-17T07:56:22.817725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815273+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:56:22.815273+0000","last_peered":"2024-04-17T07:56:22.815273+0000","last_clean":"2024-04-17T07:56:22.815273+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:56:22.815273+0000","last_undegraded":"2024-04-17T07:56:22.815273+0000","last_fullsized":"2024-04-17T07:56:22.815273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819452+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:56:22.819452+0000","last_peered":"2024-04-17T07:56:22.819452+0000","last_clean":"2024-04-17T07:56:22.819452+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:56:22.819452+0000","last_undegraded":"2024-04-17T07:56:22.819452+0000","last_fullsized":"2024-04-17T07:56:22.819452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.583770+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:56:23.583770+0000","last_peered":"2024-04-17T07:56:23.583770+0000","last_clean":"2024-04-17T07:56:23.583770+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:56:23.583770+0000","last_undegraded":"2024-04-17T07:56:23.583770+0000","last_fullsized":"2024-04-17T07:56:23.583770+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829664+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:56:22.829664+0000","last_peered":"2024-04-17T07:56:22.829664+0000","last_clean":"2024-04-17T07:56:22.829664+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:56:22.829664+0000","last_undegraded":"2024-04-17T07:56:22.829664+0000","last_fullsized":"2024-04-17T07:56:22.829664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815646+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:56:22.815646+0000","last_peered":"2024-04-17T07:56:22.815646+0000","last_clean":"2024-04-17T07:56:22.815646+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:56:22.815646+0000","last_undegraded":"2024-04-17T07:56:22.815646+0000","last_fullsized":"2024-04-17T07:56:22.815646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.828382+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:56:22.828382+0000","last_peered":"2024-04-17T07:56:22.828382+0000","last_clean":"2024-04-17T07:56:22.828382+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:56:22.828382+0000","last_undegraded":"2024-04-17T07:56:22.828382+0000","last_fullsized":"2024-04-17T07:56:22.828382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814981+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:56:22.814981+0000","last_peered":"2024-04-17T07:56:22.814981+0000","last_clean":"2024-04-17T07:56:22.814981+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:56:22.814981+0000","last_undegraded":"2024-04-17T07:56:22.814981+0000","last_fullsized":"2024-04-17T07:56:22.814981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.833566+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:56:22.833566+0000","last_peered":"2024-04-17T07:56:22.833566+0000","last_clean":"2024-04-17T07:56:22.833566+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:56:22.833566+0000","last_undegraded":"2024-04-17T07:56:22.833566+0000","last_fullsized":"2024-04-17T07:56:22.833566+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.827969+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:56:22.827969+0000","last_peered":"2024-04-17T07:56:22.827969+0000","last_clean":"2024-04-17T07:56:22.827969+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:56:22.827969+0000","last_undegraded":"2024-04-17T07:56:22.827969+0000","last_fullsized":"2024-04-17T07:56:22.827969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818969+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:56:22.818969+0000","last_peered":"2024-04-17T07:56:22.818969+0000","last_clean":"2024-04-17T07:56:22.818969+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:56:22.818969+0000","last_undegraded":"2024-04-17T07:56:22.818969+0000","last_fullsized":"2024-04-17T07:56:22.818969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:14.704280+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:57:14.704280+0000","last_peered":"2024-04-17T07:57:14.704280+0000","last_clean":"2024-04-17T07:57:14.704280+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:57:14.704280+0000","last_undegraded":"2024-04-17T07:57:14.704280+0000","last_fullsized":"2024-04-17T07:57:14.704280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.842324+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:56:22.842324+0000","last_peered":"2024-04-17T07:56:22.842324+0000","last_clean":"2024-04-17T07:56:22.842324+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:56:22.842324+0000","last_undegraded":"2024-04-17T07:56:22.842324+0000","last_fullsized":"2024-04-17T07:56:22.842324+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.833241+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:56:22.833241+0000","last_peered":"2024-04-17T07:56:22.833241+0000","last_clean":"2024-04-17T07:56:22.833241+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:56:22.833241+0000","last_undegraded":"2024-04-17T07:56:22.833241+0000","last_fullsized":"2024-04-17T07:56:22.833241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813003+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:56:22.813003+0000","last_peered":"2024-04-17T07:56:22.813003+0000","last_clean":"2024-04-17T07:56:22.813003+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:56:22.813003+0000","last_undegraded":"2024-04-17T07:56:22.813003+0000","last_fullsized":"2024-04-17T07:56:22.813003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814636+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:56:22.814636+0000","last_peered":"2024-04-17T07:56:22.814636+0000","last_clean":"2024-04-17T07:56:22.814636+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:56:22.814636+0000","last_undegraded":"2024-04-17T07:56:22.814636+0000","last_fullsized":"2024-04-17T07:56:22.814636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.825141+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:56:22.825141+0000","last_peered":"2024-04-17T07:56:22.825141+0000","last_clean":"2024-04-17T07:56:22.825141+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:56:22.825141+0000","last_undegraded":"2024-04-17T07:56:22.825141+0000","last_fullsized":"2024-04-17T07:56:22.825141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.820988+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:56:22.820988+0000","last_peered":"2024-04-17T07:56:22.820988+0000","last_clean":"2024-04-17T07:56:22.820988+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:56:22.820988+0000","last_undegraded":"2024-04-17T07:56:22.820988+0000","last_fullsized":"2024-04-17T07:56:22.820988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.828216+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:56:22.828216+0000","last_peered":"2024-04-17T07:56:22.828216+0000","last_clean":"2024-04-17T07:56:22.828216+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:56:22.828216+0000","last_undegraded":"2024-04-17T07:56:22.828216+0000","last_fullsized":"2024-04-17T07:56:22.828216+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818486+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:56:22.818486+0000","last_peered":"2024-04-17T07:56:22.818486+0000","last_clean":"2024-04-17T07:56:22.818486+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:56:22.818486+0000","last_undegraded":"2024-04-17T07:56:22.818486+0000","last_fullsized":"2024-04-17T07:56:22.818486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815143+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:56:22.815143+0000","last_peered":"2024-04-17T07:56:22.815143+0000","last_clean":"2024-04-17T07:56:22.815143+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:56:22.815143+0000","last_undegraded":"2024-04-17T07:56:22.815143+0000","last_fullsized":"2024-04-17T07:56:22.815143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829137+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:56:22.829137+0000","last_peered":"2024-04-17T07:56:22.829137+0000","last_clean":"2024-04-17T07:56:22.829137+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:56:22.829137+0000","last_undegraded":"2024-04-17T07:56:22.829137+0000","last_fullsized":"2024-04-17T07:56:22.829137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829737+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:56:22.829737+0000","last_peered":"2024-04-17T07:56:22.829737+0000","last_clean":"2024-04-17T07:56:22.829737+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:56:22.829737+0000","last_undegraded":"2024-04-17T07:56:22.829737+0000","last_fullsized":"2024-04-17T07:56:22.829737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829678+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:56:22.829678+0000","last_peered":"2024-04-17T07:56:22.829678+0000","last_clean":"2024-04-17T07:56:22.829678+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:56:22.829678+0000","last_undegraded":"2024-04-17T07:56:22.829678+0000","last_fullsized":"2024-04-17T07:56:22.829678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.835465+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:56:22.835465+0000","last_peered":"2024-04-17T07:56:22.835465+0000","last_clean":"2024-04-17T07:56:22.835465+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:56:22.835465+0000","last_undegraded":"2024-04-17T07:56:22.835465+0000","last_fullsized":"2024-04-17T07:56:22.835465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812456+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:56:22.812456+0000","last_peered":"2024-04-17T07:56:22.812456+0000","last_clean":"2024-04-17T07:56:22.812456+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:56:22.812456+0000","last_undegraded":"2024-04-17T07:56:22.812456+0000","last_fullsized":"2024-04-17T07:56:22.812456+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812861+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:56:22.812861+0000","last_peered":"2024-04-17T07:56:22.812861+0000","last_clean":"2024-04-17T07:56:22.812861+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:56:22.812861+0000","last_undegraded":"2024-04-17T07:56:22.812861+0000","last_fullsized":"2024-04-17T07:56:22.812861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817884+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:56:22.817884+0000","last_peered":"2024-04-17T07:56:22.817884+0000","last_clean":"2024-04-17T07:56:22.817884+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:56:22.817884+0000","last_undegraded":"2024-04-17T07:56:22.817884+0000","last_fullsized":"2024-04-17T07:56:22.817884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812650+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:56:22.812650+0000","last_peered":"2024-04-17T07:56:22.812650+0000","last_clean":"2024-04-17T07:56:22.812650+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:56:22.812650+0000","last_undegraded":"2024-04-17T07:56:22.812650+0000","last_fullsized":"2024-04-17T07:56:22.812650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.581803+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:56:23.581803+0000","last_peered":"2024-04-17T07:56:23.581803+0000","last_clean":"2024-04-17T07:56:23.581803+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:56:23.581803+0000","last_undegraded":"2024-04-17T07:56:23.581803+0000","last_fullsized":"2024-04-17T07:56:23.581803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814489+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:56:22.814489+0000","last_peered":"2024-04-17T07:56:22.814489+0000","last_clean":"2024-04-17T07:56:22.814489+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:56:22.814489+0000","last_undegraded":"2024-04-17T07:56:22.814489+0000","last_fullsized":"2024-04-17T07:56:22.814489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.830095+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:56:22.830095+0000","last_peered":"2024-04-17T07:56:22.830095+0000","last_clean":"2024-04-17T07:56:22.830095+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:56:22.830095+0000","last_undegraded":"2024-04-17T07:56:22.830095+0000","last_fullsized":"2024-04-17T07:56:22.830095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.831784+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:56:22.831784+0000","last_peered":"2024-04-17T07:56:22.831784+0000","last_clean":"2024-04-17T07:56:22.831784+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:56:22.831784+0000","last_undegraded":"2024-04-17T07:56:22.831784+0000","last_fullsized":"2024-04-17T07:56:22.831784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818757+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:56:22.818757+0000","last_peered":"2024-04-17T07:56:22.818757+0000","last_clean":"2024-04-17T07:56:22.818757+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:56:22.818757+0000","last_undegraded":"2024-04-17T07:56:22.818757+0000","last_fullsized":"2024-04-17T07:56:22.818757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812575+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:56:22.812575+0000","last_peered":"2024-04-17T07:56:22.812575+0000","last_clean":"2024-04-17T07:56:22.812575+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:56:22.812575+0000","last_undegraded":"2024-04-17T07:56:22.812575+0000","last_fullsized":"2024-04-17T07:56:22.812575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812702+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:56:22.812702+0000","last_peered":"2024-04-17T07:56:22.812702+0000","last_clean":"2024-04-17T07:56:22.812702+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:56:22.812702+0000","last_undegraded":"2024-04-17T07:56:22.812702+0000","last_fullsized":"2024-04-17T07:56:22.812702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.831858+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:56:22.831858+0000","last_peered":"2024-04-17T07:56:22.831858+0000","last_clean":"2024-04-17T07:56:22.831858+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:56:22.831858+0000","last_undegraded":"2024-04-17T07:56:22.831858+0000","last_fullsized":"2024-04-17T07:56:22.831858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812639+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:56:22.812639+0000","last_peered":"2024-04-17T07:56:22.812639+0000","last_clean":"2024-04-17T07:56:22.812639+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:56:22.812639+0000","last_undegraded":"2024-04-17T07:56:22.812639+0000","last_fullsized":"2024-04-17T07:56:22.812639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.825593+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:56:22.825593+0000","last_peered":"2024-04-17T07:56:22.825593+0000","last_clean":"2024-04-17T07:56:22.825593+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:56:22.825593+0000","last_undegraded":"2024-04-17T07:56:22.825593+0000","last_fullsized":"2024-04-17T07:56:22.825593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815141+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:56:22.815141+0000","last_peered":"2024-04-17T07:56:22.815141+0000","last_clean":"2024-04-17T07:56:22.815141+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:56:22.815141+0000","last_undegraded":"2024-04-17T07:56:22.815141+0000","last_fullsized":"2024-04-17T07:56:22.815141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.581895+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:56:23.581895+0000","last_peered":"2024-04-17T07:56:23.581895+0000","last_clean":"2024-04-17T07:56:23.581895+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:56:23.581895+0000","last_undegraded":"2024-04-17T07:56:23.581895+0000","last_fullsized":"2024-04-17T07:56:23.581895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813175+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:56:22.813175+0000","last_peered":"2024-04-17T07:56:22.813175+0000","last_clean":"2024-04-17T07:56:22.813175+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:56:22.813175+0000","last_undegraded":"2024-04-17T07:56:22.813175+0000","last_fullsized":"2024-04-17T07:56:22.813175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816392+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:56:22.816392+0000","last_peered":"2024-04-17T07:56:22.816392+0000","last_clean":"2024-04-17T07:56:22.816392+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:56:22.816392+0000","last_undegraded":"2024-04-17T07:56:22.816392+0000","last_fullsized":"2024-04-17T07:56:22.816392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815143+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:56:22.815143+0000","last_peered":"2024-04-17T07:56:22.815143+0000","last_clean":"2024-04-17T07:56:22.815143+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:56:22.815143+0000","last_undegraded":"2024-04-17T07:56:22.815143+0000","last_fullsized":"2024-04-17T07:56:22.815143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818113+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:56:22.818113+0000","last_peered":"2024-04-17T07:56:22.818113+0000","last_clean":"2024-04-17T07:56:22.818113+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:56:22.818113+0000","last_undegraded":"2024-04-17T07:56:22.818113+0000","last_fullsized":"2024-04-17T07:56:22.818113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815266+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:56:22.815266+0000","last_peered":"2024-04-17T07:56:22.815266+0000","last_clean":"2024-04-17T07:56:22.815266+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:56:22.815266+0000","last_undegraded":"2024-04-17T07:56:22.815266+0000","last_fullsized":"2024-04-17T07:56:22.815266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818579+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:56:22.818579+0000","last_peered":"2024-04-17T07:56:22.818579+0000","last_clean":"2024-04-17T07:56:22.818579+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:56:22.818579+0000","last_undegraded":"2024-04-17T07:56:22.818579+0000","last_fullsized":"2024-04-17T07:56:22.818579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.591665+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:56:23.591665+0000","last_peered":"2024-04-17T07:56:23.591665+0000","last_clean":"2024-04-17T07:56:23.591665+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:56:23.591665+0000","last_undegraded":"2024-04-17T07:56:23.591665+0000","last_fullsized":"2024-04-17T07:56:23.591665+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815940+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:56:22.815940+0000","last_peered":"2024-04-17T07:56:22.815940+0000","last_clean":"2024-04-17T07:56:22.815940+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:56:22.815940+0000","last_undegraded":"2024-04-17T07:56:22.815940+0000","last_fullsized":"2024-04-17T07:56:22.815940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818869+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:56:22.818869+0000","last_peered":"2024-04-17T07:56:22.818869+0000","last_clean":"2024-04-17T07:56:22.818869+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:56:22.818869+0000","last_undegraded":"2024-04-17T07:56:22.818869+0000","last_fullsized":"2024-04-17T07:56:22.818869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813214+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:56:22.813214+0000","last_peered":"2024-04-17T07:56:22.813214+0000","last_clean":"2024-04-17T07:56:22.813214+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:56:22.813214+0000","last_undegraded":"2024-04-17T07:56:22.813214+0000","last_fullsized":"2024-04-17T07:56:22.813214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812522+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:56:22.812522+0000","last_peered":"2024-04-17T07:56:22.812522+0000","last_clean":"2024-04-17T07:56:22.812522+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:56:22.812522+0000","last_undegraded":"2024-04-17T07:56:22.812522+0000","last_fullsized":"2024-04-17T07:56:22.812522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816509+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:56:22.816509+0000","last_peered":"2024-04-17T07:56:22.816509+0000","last_clean":"2024-04-17T07:56:22.816509+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:56:22.816509+0000","last_undegraded":"2024-04-17T07:56:22.816509+0000","last_fullsized":"2024-04-17T07:56:22.816509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.828872+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:56:22.828872+0000","last_peered":"2024-04-17T07:56:22.828872+0000","last_clean":"2024-04-17T07:56:22.828872+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:56:22.828872+0000","last_undegraded":"2024-04-17T07:56:22.828872+0000","last_fullsized":"2024-04-17T07:56:22.828872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.820609+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:56:22.820609+0000","last_peered":"2024-04-17T07:56:22.820609+0000","last_clean":"2024-04-17T07:56:22.820609+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:56:22.820609+0000","last_undegraded":"2024-04-17T07:56:22.820609+0000","last_fullsized":"2024-04-17T07:56:22.820609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813251+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:56:22.813251+0000","last_peered":"2024-04-17T07:56:22.813251+0000","last_clean":"2024-04-17T07:56:22.813251+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:56:22.813251+0000","last_undegraded":"2024-04-17T07:56:22.813251+0000","last_fullsized":"2024-04-17T07:56:22.813251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.835601+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:56:22.835601+0000","last_peered":"2024-04-17T07:56:22.835601+0000","last_clean":"2024-04-17T07:56:22.835601+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:56:22.835601+0000","last_undegraded":"2024-04-17T07:56:22.835601+0000","last_fullsized":"2024-04-17T07:56:22.835601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818857+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:56:22.818857+0000","last_peered":"2024-04-17T07:56:22.818857+0000","last_clean":"2024-04-17T07:56:22.818857+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:56:22.818857+0000","last_undegraded":"2024-04-17T07:56:22.818857+0000","last_fullsized":"2024-04-17T07:56:22.818857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.831983+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:56:22.831983+0000","last_peered":"2024-04-17T07:56:22.831983+0000","last_clean":"2024-04-17T07:56:22.831983+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:56:22.831983+0000","last_undegraded":"2024-04-17T07:56:22.831983+0000","last_fullsized":"2024-04-17T07:56:22.831983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582329+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:56:23.582329+0000","last_peered":"2024-04-17T07:56:23.582329+0000","last_clean":"2024-04-17T07:56:23.582329+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:56:23.582329+0000","last_undegraded":"2024-04-17T07:56:23.582329+0000","last_fullsized":"2024-04-17T07:56:23.582329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.826294+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:56:22.826294+0000","last_peered":"2024-04-17T07:56:22.826294+0000","last_clean":"2024-04-17T07:56:22.826294+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:56:22.826294+0000","last_undegraded":"2024-04-17T07:56:22.826294+0000","last_fullsized":"2024-04-17T07:56:22.826294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819291+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:56:22.819291+0000","last_peered":"2024-04-17T07:56:22.819291+0000","last_clean":"2024-04-17T07:56:22.819291+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:56:22.819291+0000","last_undegraded":"2024-04-17T07:56:22.819291+0000","last_fullsized":"2024-04-17T07:56:22.819291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67017,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":523,"num_read_kb":334,"num_write":165,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":500815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1716224,"data_stored":1151099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808879,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":505147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":505067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1097271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":496227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":558762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":497615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":560503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":496421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":501268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1638400,"data_stored":1088372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:57:21.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T07:57:21.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:20 smithi053 ceph-mon[36437]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:21.262 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:20 smithi046 ceph-mon[32038]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:21.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:20 smithi062 ceph-mon[36507]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:22.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:21 smithi046 ceph-mon[32038]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:22.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-04-17T07:57:22.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:21 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-17T07:57:22.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:21 smithi062 ceph-mon[36507]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:22.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-04-17T07:57:22.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:21 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-17T07:57:22.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:21 smithi053 ceph-mon[36437]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:22.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-04-17T07:57:22.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:21 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-17T07:57:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:22 smithi062 ceph-mon[36507]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-04-17T07:57:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:22 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-17T07:57:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:22 smithi062 ceph-mon[36507]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-17T07:57:23.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:22 smithi062 ceph-mon[36507]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T07:57:23.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:22 smithi046 ceph-mon[32038]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:23.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-04-17T07:57:23.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:22 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-17T07:57:23.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:22 smithi046 ceph-mon[32038]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-17T07:57:23.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:22 smithi046 ceph-mon[32038]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:22 smithi053 ceph-mon[36437]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-04-17T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:22 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-17T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:22 smithi053 ceph-mon[36437]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-17T07:57:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:22 smithi053 ceph-mon[36437]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T07:57:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 -- 172.21.15.46:0/4036615736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006b1f0 msgr2=0x7f7f0006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 --2- 172.21.15.46:0/4036615736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006b1f0 0x7f7f0006b650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098f0 tx=0x7f7ee8009c00 comp rx=0 tx=0).stop 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 -- 172.21.15.46:0/4036615736 shutdown_connections 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 --2- 172.21.15.46:0/4036615736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f0006bb90 0x7f7f0010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 --2- 172.21.15.46:0/4036615736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f00108a40 0x7f7f00108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 --2- 172.21.15.46:0/4036615736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006b1f0 0x7f7f0006b650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 -- 172.21.15.46:0/4036615736 >> 172.21.15.46:0/4036615736 conn(0x7f7f00078900 msgr2=0x7f7f00078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 -- 172.21.15.46:0/4036615736 shutdown_connections 2024-04-17T07:57:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.979+0000 7f7f04e00700 1 -- 172.21.15.46:0/4036615736 wait complete. 2024-04-17T07:57:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.980+0000 7f7f04e00700 1 Processor -- start 2024-04-17T07:57:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.981+0000 7f7f04e00700 1 -- start start 2024-04-17T07:57:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.981+0000 7f7f04e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0006b1f0 0x7f7f00101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.981+0000 7f7f04e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006bb90 0x7f7f00101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.981+0000 7f7f04e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f00108a40 0x7f7f001024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.981+0000 7f7f04e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00112da0 con 0x7f7f00108a40 2024-04-17T07:57:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.981+0000 7f7f04e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00112c20 con 0x7f7f0006b1f0 2024-04-17T07:57:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.981+0000 7f7f04e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00112f20 con 0x7f7f0006bb90 2024-04-17T07:57:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7ef7e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006bb90 0x7f7f00101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7efde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0006b1f0 0x7f7f00101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7efe800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f00108a40 0x7f7f001024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7efde00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0006b1f0 0x7f7f00101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56954/0 (socket says 172.21.15.46:56954) 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7ef7e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006bb90 0x7f7f00101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:59286/0 (socket says 172.21.15.46:59286) 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7efde00700 1 -- 172.21.15.46:0/3769692583 learned_addr learned my addr 172.21.15.46:0/3769692583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7efe800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f00108a40 0x7f7f001024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39282/0 (socket says 172.21.15.46:39282) 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7ef7e00700 1 -- 172.21.15.46:0/3769692583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0006b1f0 msgr2=0x7f7f00101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7ef7e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0006b1f0 0x7f7f00101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7ef7e00700 1 -- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f00108a40 msgr2=0x7f7f001024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7ef7e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f00108a40 0x7f7f001024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.982+0000 7f7ef7e00700 1 -- 172.21.15.46:0/3769692583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f001a77d0 con 0x7f7f0006bb90 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.983+0000 7f7ef7e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006bb90 0x7f7f00101f60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8000c00 tx=0x7f7ee800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.983+0000 7f7ef6a00700 1 -- 172.21.15.46:0/3769692583 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8005040 con 0x7f7f0006bb90 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.983+0000 7f7ef6a00700 1 -- 172.21.15.46:0/3769692583 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee80056c0 con 0x7f7f0006bb90 2024-04-17T07:57:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.983+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f001a7a60 con 0x7f7f0006bb90 2024-04-17T07:57:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.983+0000 7f7ef6a00700 1 -- 172.21.15.46:0/3769692583 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8003a00 con 0x7f7f0006bb90 2024-04-17T07:57:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.983+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f001a7f10 con 0x7f7f0006bb90 2024-04-17T07:57:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.984+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f00104180 con 0x7f7f0006bb90 2024-04-17T07:57:23.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.986+0000 7f7ef6a00700 1 -- 172.21.15.46:0/3769692583 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ee800b560 con 0x7f7f0006bb90 2024-04-17T07:57:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.988+0000 7f7ef6a00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7ee40a3cc0 0x7f7ee40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.988+0000 7f7ef6a00700 1 -- 172.21.15.46:0/3769692583 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7ee80d17a0 con 0x7f7f0006bb90 2024-04-17T07:57:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.988+0000 7f7efde00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7ee40a3cc0 0x7f7ee40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.989+0000 7f7efde00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7ee40a3cc0 0x7f7ee40a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ca0 tx=0x7f7ef0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:23.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:23.990+0000 7f7ef6a00700 1 -- 172.21.15.46:0/3769692583 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7ee8013030 con 0x7f7f0006bb90 2024-04-17T07:57:24.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.236+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f00062e90 con 0x7f7ee40a3cc0 2024-04-17T07:57:24.237 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:23 smithi046 ceph-mon[32038]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T07:57:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.259+0000 7f7ef6a00700 1 -- 172.21.15.46:0/3769692583 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340446 (secure 0 0 0) 0x7f7f00062e90 con 0x7f7ee40a3cc0 2024-04-17T07:57:24.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7ee40a3cc0 msgr2=0x7f7ee40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7ee40a3cc0 0x7f7ee40a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ca0 tx=0x7f7ef0009260 comp rx=0 tx=0).stop 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006bb90 msgr2=0x7f7f00101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006bb90 0x7f7f00101f60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8000c00 tx=0x7f7ee800fa00 comp rx=0 tx=0).stop 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 shutdown_connections 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7ee40a3cc0 0x7f7ee40a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f00108a40 0x7f7f001024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0006b1f0 0x7f7f00101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 --2- 172.21.15.46:0/3769692583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f0006bb90 0x7f7f00101f60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 >> 172.21.15.46:0/3769692583 conn(0x7f7f00078900 msgr2=0x7f7f000ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 shutdown_connections 2024-04-17T07:57:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:24.266+0000 7f7f04e00700 1 -- 172.21.15.46:0/3769692583 wait complete. 2024-04-17T07:57:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:57:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:23 smithi062 ceph-mon[36507]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T07:57:24.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:23 smithi053 ceph-mon[36437]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T07:57:25.092 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-17T07:57:23.555739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657868,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":680,"num_read_kb":459,"num_write":352,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":520,"ondisk_log_size":520,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379992,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561832,"statfs":{"total":1151940427776,"available":1151551315968,"internally_reserved":0,"allocated":14483456,"data_stored":7987031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004129"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.981863+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:57:21.981863+0000","last_peered":"2024-04-17T07:57:21.981863+0000","last_clean":"2024-04-17T07:57:21.981863+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:57:21.981863+0000","last_undegraded":"2024-04-17T07:57:21.981863+0000","last_fullsized":"2024-04-17T07:57:21.981863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817631+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:56:22.817631+0000","last_peered":"2024-04-17T07:56:22.817631+0000","last_clean":"2024-04-17T07:56:22.817631+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:56:22.817631+0000","last_undegraded":"2024-04-17T07:56:22.817631+0000","last_fullsized":"2024-04-17T07:56:22.817631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815576+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:56:22.815576+0000","last_peered":"2024-04-17T07:56:22.815576+0000","last_clean":"2024-04-17T07:56:22.815576+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:56:22.815576+0000","last_undegraded":"2024-04-17T07:56:22.815576+0000","last_fullsized":"2024-04-17T07:56:22.815576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812907+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:56:22.812907+0000","last_peered":"2024-04-17T07:56:22.812907+0000","last_clean":"2024-04-17T07:56:22.812907+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:56:22.812907+0000","last_undegraded":"2024-04-17T07:56:22.812907+0000","last_fullsized":"2024-04-17T07:56:22.812907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814829+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:56:22.814829+0000","last_peered":"2024-04-17T07:56:22.814829+0000","last_clean":"2024-04-17T07:56:22.814829+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:56:22.814829+0000","last_undegraded":"2024-04-17T07:56:22.814829+0000","last_fullsized":"2024-04-17T07:56:22.814829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.523497+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:57:21.523497+0000","last_peered":"2024-04-17T07:57:21.523497+0000","last_clean":"2024-04-17T07:57:21.523497+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:57:21.523497+0000","last_undegraded":"2024-04-17T07:57:21.523497+0000","last_fullsized":"2024-04-17T07:57:21.523497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.985326+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:57:21.985326+0000","last_peered":"2024-04-17T07:57:21.985326+0000","last_clean":"2024-04-17T07:57:21.985326+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:57:21.985326+0000","last_undegraded":"2024-04-17T07:57:21.985326+0000","last_fullsized":"2024-04-17T07:57:21.985326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981303+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:57:22.981303+0000","last_peered":"2024-04-17T07:57:22.981303+0000","last_clean":"2024-04-17T07:57:22.981303+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:57:22.981303+0000","last_undegraded":"2024-04-17T07:57:22.981303+0000","last_fullsized":"2024-04-17T07:57:22.981303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582310+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:56:23.582310+0000","last_peered":"2024-04-17T07:56:23.582310+0000","last_clean":"2024-04-17T07:56:23.582310+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:56:23.582310+0000","last_undegraded":"2024-04-17T07:56:23.582310+0000","last_fullsized":"2024-04-17T07:56:23.582310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.517149+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:57:21.517149+0000","last_peered":"2024-04-17T07:57:21.517149+0000","last_clean":"2024-04-17T07:57:21.517149+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:57:21.517149+0000","last_undegraded":"2024-04-17T07:57:21.517149+0000","last_fullsized":"2024-04-17T07:57:21.517149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.985436+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:57:21.985436+0000","last_peered":"2024-04-17T07:57:21.985436+0000","last_clean":"2024-04-17T07:57:21.985436+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:57:21.985436+0000","last_undegraded":"2024-04-17T07:57:21.985436+0000","last_fullsized":"2024-04-17T07:57:21.985436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812867+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:56:22.812867+0000","last_peered":"2024-04-17T07:56:22.812867+0000","last_clean":"2024-04-17T07:56:22.812867+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:56:22.812867+0000","last_undegraded":"2024-04-17T07:56:22.812867+0000","last_fullsized":"2024-04-17T07:56:22.812867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.826013+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:56:22.826013+0000","last_peered":"2024-04-17T07:56:22.826013+0000","last_clean":"2024-04-17T07:56:22.826013+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:56:22.826013+0000","last_undegraded":"2024-04-17T07:56:22.826013+0000","last_fullsized":"2024-04-17T07:56:22.826013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814974+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:56:22.814974+0000","last_peered":"2024-04-17T07:56:22.814974+0000","last_clean":"2024-04-17T07:56:22.814974+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:56:22.814974+0000","last_undegraded":"2024-04-17T07:56:22.814974+0000","last_fullsized":"2024-04-17T07:56:22.814974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816473+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:56:22.816473+0000","last_peered":"2024-04-17T07:56:22.816473+0000","last_clean":"2024-04-17T07:56:22.816473+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:56:22.816473+0000","last_undegraded":"2024-04-17T07:56:22.816473+0000","last_fullsized":"2024-04-17T07:56:22.816473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.581774+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:56:23.581774+0000","last_peered":"2024-04-17T07:56:23.581774+0000","last_clean":"2024-04-17T07:56:23.581774+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:56:23.581774+0000","last_undegraded":"2024-04-17T07:56:23.581774+0000","last_fullsized":"2024-04-17T07:56:23.581774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.981760+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:57:21.981760+0000","last_peered":"2024-04-17T07:57:21.981760+0000","last_clean":"2024-04-17T07:57:21.981760+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:57:21.981760+0000","last_undegraded":"2024-04-17T07:57:21.981760+0000","last_fullsized":"2024-04-17T07:57:21.981760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829471+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:56:22.829471+0000","last_peered":"2024-04-17T07:56:22.829471+0000","last_clean":"2024-04-17T07:56:22.829471+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:56:22.829471+0000","last_undegraded":"2024-04-17T07:56:22.829471+0000","last_fullsized":"2024-04-17T07:56:22.829471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.523989+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:57:21.523989+0000","last_peered":"2024-04-17T07:57:21.523989+0000","last_clean":"2024-04-17T07:57:21.523989+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:57:21.523989+0000","last_undegraded":"2024-04-17T07:57:21.523989+0000","last_fullsized":"2024-04-17T07:57:21.523989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812665+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:56:22.812665+0000","last_peered":"2024-04-17T07:56:22.812665+0000","last_clean":"2024-04-17T07:56:22.812665+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:56:22.812665+0000","last_undegraded":"2024-04-17T07:56:22.812665+0000","last_fullsized":"2024-04-17T07:56:22.812665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.835085+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:56:22.835085+0000","last_peered":"2024-04-17T07:56:22.835085+0000","last_clean":"2024-04-17T07:56:22.835085+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:56:22.835085+0000","last_undegraded":"2024-04-17T07:56:22.835085+0000","last_fullsized":"2024-04-17T07:56:22.835085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.834799+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:56:22.834799+0000","last_peered":"2024-04-17T07:56:22.834799+0000","last_clean":"2024-04-17T07:56:22.834799+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:56:22.834799+0000","last_undegraded":"2024-04-17T07:56:22.834799+0000","last_fullsized":"2024-04-17T07:56:22.834799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812293+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:56:22.812293+0000","last_peered":"2024-04-17T07:56:22.812293+0000","last_clean":"2024-04-17T07:56:22.812293+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:56:22.812293+0000","last_undegraded":"2024-04-17T07:56:22.812293+0000","last_fullsized":"2024-04-17T07:56:22.812293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984474+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:57:22.984474+0000","last_peered":"2024-04-17T07:57:22.984474+0000","last_clean":"2024-04-17T07:57:22.984474+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:57:22.984474+0000","last_undegraded":"2024-04-17T07:57:22.984474+0000","last_fullsized":"2024-04-17T07:57:22.984474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816959+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:56:22.816959+0000","last_peered":"2024-04-17T07:56:22.816959+0000","last_clean":"2024-04-17T07:56:22.816959+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:56:22.816959+0000","last_undegraded":"2024-04-17T07:56:22.816959+0000","last_fullsized":"2024-04-17T07:56:22.816959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812385+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:56:22.812385+0000","last_peered":"2024-04-17T07:56:22.812385+0000","last_clean":"2024-04-17T07:56:22.812385+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:56:22.812385+0000","last_undegraded":"2024-04-17T07:56:22.812385+0000","last_fullsized":"2024-04-17T07:56:22.812385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2024-04-17T07:55:32.541703+0000","last_change":"2024-04-17T07:57:22.131751+0000","last_active":"2024-04-17T07:57:22.131751+0000","last_peered":"2024-04-17T07:55:32.541703+0000","last_clean":"2024-04-17T07:55:32.541703+0000","last_became_active":"2024-04-17T07:55:25.508793+0000","last_became_peered":"2024-04-17T07:55:25.508793+0000","last_unstale":"2024-04-17T07:55:32.541703+0000","last_undegraded":"2024-04-17T07:55:32.541703+0000","last_fullsized":"2024-04-17T07:55:32.541703+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582292+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:56:23.582292+0000","last_peered":"2024-04-17T07:56:23.582292+0000","last_clean":"2024-04-17T07:56:23.582292+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:56:23.582292+0000","last_undegraded":"2024-04-17T07:56:23.582292+0000","last_fullsized":"2024-04-17T07:56:23.582292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812345+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:56:22.812345+0000","last_peered":"2024-04-17T07:56:22.812345+0000","last_clean":"2024-04-17T07:56:22.812345+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:56:22.812345+0000","last_undegraded":"2024-04-17T07:56:22.812345+0000","last_fullsized":"2024-04-17T07:56:22.812345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814627+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:56:22.814627+0000","last_peered":"2024-04-17T07:56:22.814627+0000","last_clean":"2024-04-17T07:56:22.814627+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:56:22.814627+0000","last_undegraded":"2024-04-17T07:56:22.814627+0000","last_fullsized":"2024-04-17T07:56:22.814627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818056+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:56:22.818056+0000","last_peered":"2024-04-17T07:56:22.818056+0000","last_clean":"2024-04-17T07:56:22.818056+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:56:22.818056+0000","last_undegraded":"2024-04-17T07:56:22.818056+0000","last_fullsized":"2024-04-17T07:56:22.818056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813003+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:56:22.813003+0000","last_peered":"2024-04-17T07:56:22.813003+0000","last_clean":"2024-04-17T07:56:22.813003+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:56:22.813003+0000","last_undegraded":"2024-04-17T07:56:22.813003+0000","last_fullsized":"2024-04-17T07:56:22.813003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:19.670610+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:57:19.670610+0000","last_peered":"2024-04-17T07:57:19.670610+0000","last_clean":"2024-04-17T07:57:19.670610+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:57:19.670610+0000","last_undegraded":"2024-04-17T07:57:19.670610+0000","last_fullsized":"2024-04-17T07:57:19.670610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815118+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:56:22.815118+0000","last_peered":"2024-04-17T07:56:22.815118+0000","last_clean":"2024-04-17T07:56:22.815118+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:56:22.815118+0000","last_undegraded":"2024-04-17T07:56:22.815118+0000","last_fullsized":"2024-04-17T07:56:22.815118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812838+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:56:22.812838+0000","last_peered":"2024-04-17T07:56:22.812838+0000","last_clean":"2024-04-17T07:56:22.812838+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:56:22.812838+0000","last_undegraded":"2024-04-17T07:56:22.812838+0000","last_fullsized":"2024-04-17T07:56:22.812838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815586+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:56:22.815586+0000","last_peered":"2024-04-17T07:56:22.815586+0000","last_clean":"2024-04-17T07:56:22.815586+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:56:22.815586+0000","last_undegraded":"2024-04-17T07:56:22.815586+0000","last_fullsized":"2024-04-17T07:56:22.815586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818071+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:56:22.818071+0000","last_peered":"2024-04-17T07:56:22.818071+0000","last_clean":"2024-04-17T07:56:22.818071+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:56:22.818071+0000","last_undegraded":"2024-04-17T07:56:22.818071+0000","last_fullsized":"2024-04-17T07:56:22.818071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.981634+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:57:21.981634+0000","last_peered":"2024-04-17T07:57:21.981634+0000","last_clean":"2024-04-17T07:57:21.981634+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:57:21.981634+0000","last_undegraded":"2024-04-17T07:57:21.981634+0000","last_fullsized":"2024-04-17T07:57:21.981634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.824775+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:56:22.824775+0000","last_peered":"2024-04-17T07:56:22.824775+0000","last_clean":"2024-04-17T07:56:22.824775+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:56:22.824775+0000","last_undegraded":"2024-04-17T07:56:22.824775+0000","last_fullsized":"2024-04-17T07:56:22.824775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.980936+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:57:22.980936+0000","last_peered":"2024-04-17T07:57:22.980936+0000","last_clean":"2024-04-17T07:57:22.980936+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:57:22.980936+0000","last_undegraded":"2024-04-17T07:57:22.980936+0000","last_fullsized":"2024-04-17T07:57:22.980936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817465+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:56:22.817465+0000","last_peered":"2024-04-17T07:56:22.817465+0000","last_clean":"2024-04-17T07:56:22.817465+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:56:22.817465+0000","last_undegraded":"2024-04-17T07:56:22.817465+0000","last_fullsized":"2024-04-17T07:56:22.817465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.819160+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:56:22.819160+0000","last_peered":"2024-04-17T07:56:22.819160+0000","last_clean":"2024-04-17T07:56:22.819160+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:56:22.819160+0000","last_undegraded":"2024-04-17T07:56:22.819160+0000","last_fullsized":"2024-04-17T07:56:22.819160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.827202+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:56:22.827202+0000","last_peered":"2024-04-17T07:56:22.827202+0000","last_clean":"2024-04-17T07:56:22.827202+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:56:22.827202+0000","last_undegraded":"2024-04-17T07:56:22.827202+0000","last_fullsized":"2024-04-17T07:56:22.827202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981197+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:57:22.981197+0000","last_peered":"2024-04-17T07:57:22.981197+0000","last_clean":"2024-04-17T07:57:22.981197+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:57:22.981197+0000","last_undegraded":"2024-04-17T07:57:22.981197+0000","last_fullsized":"2024-04-17T07:57:22.981197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":119,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.466994+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:57:21.466994+0000","last_peered":"2024-04-17T07:57:21.466994+0000","last_clean":"2024-04-17T07:57:21.466994+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:57:21.466994+0000","last_undegraded":"2024-04-17T07:57:21.466994+0000","last_fullsized":"2024-04-17T07:57:21.466994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582435+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:56:23.582435+0000","last_peered":"2024-04-17T07:56:23.582435+0000","last_clean":"2024-04-17T07:56:23.582435+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:56:23.582435+0000","last_undegraded":"2024-04-17T07:56:23.582435+0000","last_fullsized":"2024-04-17T07:56:23.582435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.828709+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:56:22.828709+0000","last_peered":"2024-04-17T07:56:22.828709+0000","last_clean":"2024-04-17T07:56:22.828709+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:56:22.828709+0000","last_undegraded":"2024-04-17T07:56:22.828709+0000","last_fullsized":"2024-04-17T07:56:22.828709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.984421+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:57:21.984421+0000","last_peered":"2024-04-17T07:57:21.984421+0000","last_clean":"2024-04-17T07:57:21.984421+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:57:21.984421+0000","last_undegraded":"2024-04-17T07:57:21.984421+0000","last_fullsized":"2024-04-17T07:57:21.984421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816876+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:56:22.816876+0000","last_peered":"2024-04-17T07:56:22.816876+0000","last_clean":"2024-04-17T07:56:22.816876+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:56:22.816876+0000","last_undegraded":"2024-04-17T07:56:22.816876+0000","last_fullsized":"2024-04-17T07:56:22.816876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813057+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:56:22.813057+0000","last_peered":"2024-04-17T07:56:22.813057+0000","last_clean":"2024-04-17T07:56:22.813057+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:56:22.813057+0000","last_undegraded":"2024-04-17T07:56:22.813057+0000","last_fullsized":"2024-04-17T07:56:22.813057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817917+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:56:22.817917+0000","last_peered":"2024-04-17T07:56:22.817917+0000","last_clean":"2024-04-17T07:56:22.817917+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:56:22.817917+0000","last_undegraded":"2024-04-17T07:56:22.817917+0000","last_fullsized":"2024-04-17T07:56:22.817917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582424+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:56:23.582424+0000","last_peered":"2024-04-17T07:56:23.582424+0000","last_clean":"2024-04-17T07:56:23.582424+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:56:23.582424+0000","last_undegraded":"2024-04-17T07:56:23.582424+0000","last_fullsized":"2024-04-17T07:56:23.582424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.830785+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:56:22.830785+0000","last_peered":"2024-04-17T07:56:22.830785+0000","last_clean":"2024-04-17T07:56:22.830785+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:56:22.830785+0000","last_undegraded":"2024-04-17T07:56:22.830785+0000","last_fullsized":"2024-04-17T07:56:22.830785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":247,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.673401+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:57:21.673401+0000","last_peered":"2024-04-17T07:57:21.673401+0000","last_clean":"2024-04-17T07:57:21.673401+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:57:21.673401+0000","last_undegraded":"2024-04-17T07:57:21.673401+0000","last_fullsized":"2024-04-17T07:57:21.673401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":185,"num_read_kb":155,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815354+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:56:22.815354+0000","last_peered":"2024-04-17T07:56:22.815354+0000","last_clean":"2024-04-17T07:56:22.815354+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:56:22.815354+0000","last_undegraded":"2024-04-17T07:56:22.815354+0000","last_fullsized":"2024-04-17T07:56:22.815354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818273+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:56:22.818273+0000","last_peered":"2024-04-17T07:56:22.818273+0000","last_clean":"2024-04-17T07:56:22.818273+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:56:22.818273+0000","last_undegraded":"2024-04-17T07:56:22.818273+0000","last_fullsized":"2024-04-17T07:56:22.818273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812853+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:56:22.812853+0000","last_peered":"2024-04-17T07:56:22.812853+0000","last_clean":"2024-04-17T07:56:22.812853+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:56:22.812853+0000","last_undegraded":"2024-04-17T07:56:22.812853+0000","last_fullsized":"2024-04-17T07:56:22.812853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.825904+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:56:22.825904+0000","last_peered":"2024-04-17T07:56:22.825904+0000","last_clean":"2024-04-17T07:56:22.825904+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:56:22.825904+0000","last_undegraded":"2024-04-17T07:56:22.825904+0000","last_fullsized":"2024-04-17T07:56:22.825904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.980759+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:57:21.980759+0000","last_peered":"2024-04-17T07:57:21.980759+0000","last_clean":"2024-04-17T07:57:21.980759+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:57:21.980759+0000","last_undegraded":"2024-04-17T07:57:21.980759+0000","last_fullsized":"2024-04-17T07:57:21.980759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817949+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:56:22.817949+0000","last_peered":"2024-04-17T07:56:22.817949+0000","last_clean":"2024-04-17T07:56:22.817949+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:56:22.817949+0000","last_undegraded":"2024-04-17T07:56:22.817949+0000","last_fullsized":"2024-04-17T07:56:22.817949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.984497+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:57:21.984497+0000","last_peered":"2024-04-17T07:57:21.984497+0000","last_clean":"2024-04-17T07:57:21.984497+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:57:21.984497+0000","last_undegraded":"2024-04-17T07:57:21.984497+0000","last_fullsized":"2024-04-17T07:57:21.984497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829626+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:56:22.829626+0000","last_peered":"2024-04-17T07:56:22.829626+0000","last_clean":"2024-04-17T07:56:22.829626+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:56:22.829626+0000","last_undegraded":"2024-04-17T07:56:22.829626+0000","last_fullsized":"2024-04-17T07:56:22.829626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:19.716170+0000","last_change":"2024-04-17T07:55:25.507054+0000","last_active":"2024-04-17T07:57:19.716170+0000","last_peered":"2024-04-17T07:57:19.716170+0000","last_clean":"2024-04-17T07:57:19.716170+0000","last_became_active":"2024-04-17T07:55:25.506709+0000","last_became_peered":"2024-04-17T07:55:25.506709+0000","last_unstale":"2024-04-17T07:57:19.716170+0000","last_undegraded":"2024-04-17T07:57:19.716170+0000","last_fullsized":"2024-04-17T07:57:19.716170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:53:29.194193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.984028+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:57:21.984028+0000","last_peered":"2024-04-17T07:57:21.984028+0000","last_clean":"2024-04-17T07:57:21.984028+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:57:21.984028+0000","last_undegraded":"2024-04-17T07:57:21.984028+0000","last_fullsized":"2024-04-17T07:57:21.984028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.837796+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:56:22.837796+0000","last_peered":"2024-04-17T07:56:22.837796+0000","last_clean":"2024-04-17T07:56:22.837796+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:56:22.837796+0000","last_undegraded":"2024-04-17T07:56:22.837796+0000","last_fullsized":"2024-04-17T07:56:22.837796+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814480+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:56:22.814480+0000","last_peered":"2024-04-17T07:56:22.814480+0000","last_clean":"2024-04-17T07:56:22.814480+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:56:22.814480+0000","last_undegraded":"2024-04-17T07:56:22.814480+0000","last_fullsized":"2024-04-17T07:56:22.814480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817885+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:56:22.817885+0000","last_peered":"2024-04-17T07:56:22.817885+0000","last_clean":"2024-04-17T07:56:22.817885+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:56:22.817885+0000","last_undegraded":"2024-04-17T07:56:22.817885+0000","last_fullsized":"2024-04-17T07:56:22.817885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817725+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:56:22.817725+0000","last_peered":"2024-04-17T07:56:22.817725+0000","last_clean":"2024-04-17T07:56:22.817725+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:56:22.817725+0000","last_undegraded":"2024-04-17T07:56:22.817725+0000","last_fullsized":"2024-04-17T07:56:22.817725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815273+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:56:22.815273+0000","last_peered":"2024-04-17T07:56:22.815273+0000","last_clean":"2024-04-17T07:56:22.815273+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:56:22.815273+0000","last_undegraded":"2024-04-17T07:56:22.815273+0000","last_fullsized":"2024-04-17T07:56:22.815273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.980491+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:57:21.980491+0000","last_peered":"2024-04-17T07:57:21.980491+0000","last_clean":"2024-04-17T07:57:21.980491+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:57:21.980491+0000","last_undegraded":"2024-04-17T07:57:21.980491+0000","last_fullsized":"2024-04-17T07:57:21.980491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.583770+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:56:23.583770+0000","last_peered":"2024-04-17T07:56:23.583770+0000","last_clean":"2024-04-17T07:56:23.583770+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:56:23.583770+0000","last_undegraded":"2024-04-17T07:56:23.583770+0000","last_fullsized":"2024-04-17T07:56:23.583770+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829664+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:56:22.829664+0000","last_peered":"2024-04-17T07:56:22.829664+0000","last_clean":"2024-04-17T07:56:22.829664+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:56:22.829664+0000","last_undegraded":"2024-04-17T07:56:22.829664+0000","last_fullsized":"2024-04-17T07:56:22.829664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.533719+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:57:21.533719+0000","last_peered":"2024-04-17T07:57:21.533719+0000","last_clean":"2024-04-17T07:57:21.533719+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:57:21.533719+0000","last_undegraded":"2024-04-17T07:57:21.533719+0000","last_fullsized":"2024-04-17T07:57:21.533719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.828382+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:56:22.828382+0000","last_peered":"2024-04-17T07:56:22.828382+0000","last_clean":"2024-04-17T07:56:22.828382+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:56:22.828382+0000","last_undegraded":"2024-04-17T07:56:22.828382+0000","last_fullsized":"2024-04-17T07:56:22.828382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.984619+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:57:21.984619+0000","last_peered":"2024-04-17T07:57:21.984619+0000","last_clean":"2024-04-17T07:57:21.984619+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:57:21.984619+0000","last_undegraded":"2024-04-17T07:57:21.984619+0000","last_fullsized":"2024-04-17T07:57:21.984619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.523774+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:57:21.523774+0000","last_peered":"2024-04-17T07:57:21.523774+0000","last_clean":"2024-04-17T07:57:21.523774+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:57:21.523774+0000","last_undegraded":"2024-04-17T07:57:21.523774+0000","last_fullsized":"2024-04-17T07:57:21.523774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.827969+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:56:22.827969+0000","last_peered":"2024-04-17T07:56:22.827969+0000","last_clean":"2024-04-17T07:56:22.827969+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:56:22.827969+0000","last_undegraded":"2024-04-17T07:56:22.827969+0000","last_fullsized":"2024-04-17T07:56:22.827969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818969+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:56:22.818969+0000","last_peered":"2024-04-17T07:56:22.818969+0000","last_clean":"2024-04-17T07:56:22.818969+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:56:22.818969+0000","last_undegraded":"2024-04-17T07:56:22.818969+0000","last_fullsized":"2024-04-17T07:56:22.818969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:19.715064+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:57:19.715064+0000","last_peered":"2024-04-17T07:57:19.715064+0000","last_clean":"2024-04-17T07:57:19.715064+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:57:19.715064+0000","last_undegraded":"2024-04-17T07:57:19.715064+0000","last_fullsized":"2024-04-17T07:57:19.715064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.985224+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:57:21.985224+0000","last_peered":"2024-04-17T07:57:21.985224+0000","last_clean":"2024-04-17T07:57:21.985224+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:57:21.985224+0000","last_undegraded":"2024-04-17T07:57:21.985224+0000","last_fullsized":"2024-04-17T07:57:21.985224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982533+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:57:22.982533+0000","last_peered":"2024-04-17T07:57:22.982533+0000","last_clean":"2024-04-17T07:57:22.982533+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:57:22.982533+0000","last_undegraded":"2024-04-17T07:57:22.982533+0000","last_fullsized":"2024-04-17T07:57:22.982533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813003+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:56:22.813003+0000","last_peered":"2024-04-17T07:56:22.813003+0000","last_clean":"2024-04-17T07:56:22.813003+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:56:22.813003+0000","last_undegraded":"2024-04-17T07:56:22.813003+0000","last_fullsized":"2024-04-17T07:56:22.813003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":354,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.983665+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:57:21.983665+0000","last_peered":"2024-04-17T07:57:21.983665+0000","last_clean":"2024-04-17T07:57:21.983665+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:57:21.983665+0000","last_undegraded":"2024-04-17T07:57:21.983665+0000","last_fullsized":"2024-04-17T07:57:21.983665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.825141+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:56:22.825141+0000","last_peered":"2024-04-17T07:56:22.825141+0000","last_clean":"2024-04-17T07:56:22.825141+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:56:22.825141+0000","last_undegraded":"2024-04-17T07:56:22.825141+0000","last_fullsized":"2024-04-17T07:56:22.825141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.820988+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:56:22.820988+0000","last_peered":"2024-04-17T07:56:22.820988+0000","last_clean":"2024-04-17T07:56:22.820988+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:56:22.820988+0000","last_undegraded":"2024-04-17T07:56:22.820988+0000","last_fullsized":"2024-04-17T07:56:22.820988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.983780+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:57:21.983780+0000","last_peered":"2024-04-17T07:57:21.983780+0000","last_clean":"2024-04-17T07:57:21.983780+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:57:21.983780+0000","last_undegraded":"2024-04-17T07:57:21.983780+0000","last_fullsized":"2024-04-17T07:57:21.983780+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.980265+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:57:21.980265+0000","last_peered":"2024-04-17T07:57:21.980265+0000","last_clean":"2024-04-17T07:57:21.980265+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:57:21.980265+0000","last_undegraded":"2024-04-17T07:57:21.980265+0000","last_fullsized":"2024-04-17T07:57:21.980265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815143+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:56:22.815143+0000","last_peered":"2024-04-17T07:56:22.815143+0000","last_clean":"2024-04-17T07:56:22.815143+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:56:22.815143+0000","last_undegraded":"2024-04-17T07:56:22.815143+0000","last_fullsized":"2024-04-17T07:56:22.815143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829137+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:56:22.829137+0000","last_peered":"2024-04-17T07:56:22.829137+0000","last_clean":"2024-04-17T07:56:22.829137+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:56:22.829137+0000","last_undegraded":"2024-04-17T07:56:22.829137+0000","last_fullsized":"2024-04-17T07:56:22.829137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.980943+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:57:22.980943+0000","last_peered":"2024-04-17T07:57:22.980943+0000","last_clean":"2024-04-17T07:57:22.980943+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:57:22.980943+0000","last_undegraded":"2024-04-17T07:57:22.980943+0000","last_fullsized":"2024-04-17T07:57:22.980943+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.829678+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:56:22.829678+0000","last_peered":"2024-04-17T07:56:22.829678+0000","last_clean":"2024-04-17T07:56:22.829678+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:56:22.829678+0000","last_undegraded":"2024-04-17T07:56:22.829678+0000","last_fullsized":"2024-04-17T07:56:22.829678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984112+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:57:22.984112+0000","last_peered":"2024-04-17T07:57:22.984112+0000","last_clean":"2024-04-17T07:57:22.984112+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:57:22.984112+0000","last_undegraded":"2024-04-17T07:57:22.984112+0000","last_fullsized":"2024-04-17T07:57:22.984112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812456+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:56:22.812456+0000","last_peered":"2024-04-17T07:56:22.812456+0000","last_clean":"2024-04-17T07:56:22.812456+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:56:22.812456+0000","last_undegraded":"2024-04-17T07:56:22.812456+0000","last_fullsized":"2024-04-17T07:56:22.812456+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812861+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:56:22.812861+0000","last_peered":"2024-04-17T07:56:22.812861+0000","last_clean":"2024-04-17T07:56:22.812861+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:56:22.812861+0000","last_undegraded":"2024-04-17T07:56:22.812861+0000","last_fullsized":"2024-04-17T07:56:22.812861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.817884+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:56:22.817884+0000","last_peered":"2024-04-17T07:56:22.817884+0000","last_clean":"2024-04-17T07:56:22.817884+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:56:22.817884+0000","last_undegraded":"2024-04-17T07:56:22.817884+0000","last_fullsized":"2024-04-17T07:56:22.817884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812650+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:56:22.812650+0000","last_peered":"2024-04-17T07:56:22.812650+0000","last_clean":"2024-04-17T07:56:22.812650+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:56:22.812650+0000","last_undegraded":"2024-04-17T07:56:22.812650+0000","last_fullsized":"2024-04-17T07:56:22.812650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.581803+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:56:23.581803+0000","last_peered":"2024-04-17T07:56:23.581803+0000","last_clean":"2024-04-17T07:56:23.581803+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:56:23.581803+0000","last_undegraded":"2024-04-17T07:56:23.581803+0000","last_fullsized":"2024-04-17T07:56:23.581803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.814489+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:56:22.814489+0000","last_peered":"2024-04-17T07:56:22.814489+0000","last_clean":"2024-04-17T07:56:22.814489+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:56:22.814489+0000","last_undegraded":"2024-04-17T07:56:22.814489+0000","last_fullsized":"2024-04-17T07:56:22.814489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.830095+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:56:22.830095+0000","last_peered":"2024-04-17T07:56:22.830095+0000","last_clean":"2024-04-17T07:56:22.830095+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:56:22.830095+0000","last_undegraded":"2024-04-17T07:56:22.830095+0000","last_fullsized":"2024-04-17T07:56:22.830095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981526+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:57:22.981526+0000","last_peered":"2024-04-17T07:57:22.981526+0000","last_clean":"2024-04-17T07:57:22.981526+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:57:22.981526+0000","last_undegraded":"2024-04-17T07:57:22.981526+0000","last_fullsized":"2024-04-17T07:57:22.981526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.981123+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:57:21.981123+0000","last_peered":"2024-04-17T07:57:21.981123+0000","last_clean":"2024-04-17T07:57:21.981123+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:57:21.981123+0000","last_undegraded":"2024-04-17T07:57:21.981123+0000","last_fullsized":"2024-04-17T07:57:21.981123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812575+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:56:22.812575+0000","last_peered":"2024-04-17T07:56:22.812575+0000","last_clean":"2024-04-17T07:56:22.812575+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:56:22.812575+0000","last_undegraded":"2024-04-17T07:56:22.812575+0000","last_fullsized":"2024-04-17T07:56:22.812575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812702+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:56:22.812702+0000","last_peered":"2024-04-17T07:56:22.812702+0000","last_clean":"2024-04-17T07:56:22.812702+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:56:22.812702+0000","last_undegraded":"2024-04-17T07:56:22.812702+0000","last_fullsized":"2024-04-17T07:56:22.812702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983499+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:57:22.983499+0000","last_peered":"2024-04-17T07:57:22.983499+0000","last_clean":"2024-04-17T07:57:22.983499+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:57:22.983499+0000","last_undegraded":"2024-04-17T07:57:22.983499+0000","last_fullsized":"2024-04-17T07:57:22.983499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.812639+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:56:22.812639+0000","last_peered":"2024-04-17T07:56:22.812639+0000","last_clean":"2024-04-17T07:56:22.812639+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:56:22.812639+0000","last_undegraded":"2024-04-17T07:56:22.812639+0000","last_fullsized":"2024-04-17T07:56:22.812639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.825593+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:56:22.825593+0000","last_peered":"2024-04-17T07:56:22.825593+0000","last_clean":"2024-04-17T07:56:22.825593+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:56:22.825593+0000","last_undegraded":"2024-04-17T07:56:22.825593+0000","last_fullsized":"2024-04-17T07:56:22.825593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.984737+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:57:21.984737+0000","last_peered":"2024-04-17T07:57:21.984737+0000","last_clean":"2024-04-17T07:57:21.984737+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:57:21.984737+0000","last_undegraded":"2024-04-17T07:57:21.984737+0000","last_fullsized":"2024-04-17T07:57:21.984737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.581895+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:56:23.581895+0000","last_peered":"2024-04-17T07:56:23.581895+0000","last_clean":"2024-04-17T07:56:23.581895+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:56:23.581895+0000","last_undegraded":"2024-04-17T07:56:23.581895+0000","last_fullsized":"2024-04-17T07:56:23.581895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.813175+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:56:22.813175+0000","last_peered":"2024-04-17T07:56:22.813175+0000","last_clean":"2024-04-17T07:56:22.813175+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:56:22.813175+0000","last_undegraded":"2024-04-17T07:56:22.813175+0000","last_fullsized":"2024-04-17T07:56:22.813175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.985553+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:57:21.985553+0000","last_peered":"2024-04-17T07:57:21.985553+0000","last_clean":"2024-04-17T07:57:21.985553+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:57:21.985553+0000","last_undegraded":"2024-04-17T07:57:21.985553+0000","last_fullsized":"2024-04-17T07:57:21.985553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.815143+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:56:22.815143+0000","last_peered":"2024-04-17T07:56:22.815143+0000","last_clean":"2024-04-17T07:56:22.815143+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:56:22.815143+0000","last_undegraded":"2024-04-17T07:56:22.815143+0000","last_fullsized":"2024-04-17T07:56:22.815143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818113+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:56:22.818113+0000","last_peered":"2024-04-17T07:56:22.818113+0000","last_clean":"2024-04-17T07:56:22.818113+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:56:22.818113+0000","last_undegraded":"2024-04-17T07:56:22.818113+0000","last_fullsized":"2024-04-17T07:56:22.818113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.984201+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:57:21.984201+0000","last_peered":"2024-04-17T07:57:21.984201+0000","last_clean":"2024-04-17T07:57:21.984201+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:57:21.984201+0000","last_undegraded":"2024-04-17T07:57:21.984201+0000","last_fullsized":"2024-04-17T07:57:21.984201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818579+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:56:22.818579+0000","last_peered":"2024-04-17T07:56:22.818579+0000","last_clean":"2024-04-17T07:56:22.818579+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:56:22.818579+0000","last_undegraded":"2024-04-17T07:56:22.818579+0000","last_fullsized":"2024-04-17T07:56:22.818579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.591665+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:56:23.591665+0000","last_peered":"2024-04-17T07:56:23.591665+0000","last_clean":"2024-04-17T07:56:23.591665+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:56:23.591665+0000","last_undegraded":"2024-04-17T07:56:23.591665+0000","last_fullsized":"2024-04-17T07:56:23.591665+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.984209+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:57:21.984209+0000","last_peered":"2024-04-17T07:57:21.984209+0000","last_clean":"2024-04-17T07:57:21.984209+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:57:21.984209+0000","last_undegraded":"2024-04-17T07:57:21.984209+0000","last_fullsized":"2024-04-17T07:57:21.984209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.981278+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:57:21.981278+0000","last_peered":"2024-04-17T07:57:21.981278+0000","last_clean":"2024-04-17T07:57:21.981278+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:57:21.981278+0000","last_undegraded":"2024-04-17T07:57:21.981278+0000","last_fullsized":"2024-04-17T07:57:21.981278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.517386+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:57:21.517386+0000","last_peered":"2024-04-17T07:57:21.517386+0000","last_clean":"2024-04-17T07:57:21.517386+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:57:21.517386+0000","last_undegraded":"2024-04-17T07:57:21.517386+0000","last_fullsized":"2024-04-17T07:57:21.517386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.503010+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:57:21.503010+0000","last_peered":"2024-04-17T07:57:21.503010+0000","last_clean":"2024-04-17T07:57:21.503010+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:57:21.503010+0000","last_undegraded":"2024-04-17T07:57:21.503010+0000","last_fullsized":"2024-04-17T07:57:21.503010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.816509+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:56:22.816509+0000","last_peered":"2024-04-17T07:56:22.816509+0000","last_clean":"2024-04-17T07:56:22.816509+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:56:22.816509+0000","last_undegraded":"2024-04-17T07:56:22.816509+0000","last_fullsized":"2024-04-17T07:56:22.816509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.828872+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:56:22.828872+0000","last_peered":"2024-04-17T07:56:22.828872+0000","last_clean":"2024-04-17T07:56:22.828872+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:56:22.828872+0000","last_undegraded":"2024-04-17T07:56:22.828872+0000","last_fullsized":"2024-04-17T07:56:22.828872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.981098+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:57:21.981098+0000","last_peered":"2024-04-17T07:57:21.981098+0000","last_clean":"2024-04-17T07:57:21.981098+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:57:21.981098+0000","last_undegraded":"2024-04-17T07:57:21.981098+0000","last_fullsized":"2024-04-17T07:57:21.981098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.508216+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:57:21.508216+0000","last_peered":"2024-04-17T07:57:21.508216+0000","last_clean":"2024-04-17T07:57:21.508216+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:57:21.508216+0000","last_undegraded":"2024-04-17T07:57:21.508216+0000","last_fullsized":"2024-04-17T07:57:21.508216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984574+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:57:22.984574+0000","last_peered":"2024-04-17T07:57:22.984574+0000","last_clean":"2024-04-17T07:57:22.984574+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:57:22.984574+0000","last_undegraded":"2024-04-17T07:57:22.984574+0000","last_fullsized":"2024-04-17T07:57:22.984574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.818857+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:56:22.818857+0000","last_peered":"2024-04-17T07:56:22.818857+0000","last_clean":"2024-04-17T07:56:22.818857+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:56:22.818857+0000","last_undegraded":"2024-04-17T07:56:22.818857+0000","last_fullsized":"2024-04-17T07:56:22.818857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981814+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:57:22.981814+0000","last_peered":"2024-04-17T07:57:22.981814+0000","last_clean":"2024-04-17T07:57:22.981814+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:57:22.981814+0000","last_undegraded":"2024-04-17T07:57:22.981814+0000","last_fullsized":"2024-04-17T07:57:22.981814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:23.582329+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:56:23.582329+0000","last_peered":"2024-04-17T07:56:23.582329+0000","last_clean":"2024-04-17T07:56:23.582329+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:56:23.582329+0000","last_undegraded":"2024-04-17T07:56:23.582329+0000","last_fullsized":"2024-04-17T07:56:23.582329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:56:22.826294+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:56:22.826294+0000","last_peered":"2024-04-17T07:56:22.826294+0000","last_clean":"2024-04-17T07:56:22.826294+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:56:22.826294+0000","last_undegraded":"2024-04-17T07:56:22.826294+0000","last_fullsized":"2024-04-17T07:56:22.826294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-17T07:57:21.980940+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:57:21.980940+0000","last_peered":"2024-04-17T07:57:21.980940+0000","last_clean":"2024-04-17T07:57:21.980940+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:57:21.980940+0000","last_undegraded":"2024-04-17T07:57:21.980940+0000","last_fullsized":"2024-04-17T07:57:21.980940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67266,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":554,"num_read_kb":350,"num_write":169,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":500815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1716224,"data_stored":1151099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":505147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":505067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332156,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1106961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":496227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":558762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":497615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":560503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":501268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1638400,"data_stored":1088372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:57:25.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T07:57:25.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:24 smithi062 ceph-mon[36507]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-17T07:57:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:24 smithi062 ceph-mon[36507]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-17T07:57:25.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:24 smithi062 ceph-mon[36507]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:25.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:24 smithi046 ceph-mon[32038]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-17T07:57:25.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:24 smithi046 ceph-mon[32038]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-17T07:57:25.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:24 smithi046 ceph-mon[32038]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:24 smithi053 ceph-mon[36437]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-17T07:57:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:24 smithi053 ceph-mon[36437]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-17T07:57:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:24 smithi053 ceph-mon[36437]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:27.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:27 smithi046 ceph-mon[32038]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-17T07:57:27.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:27 smithi062 ceph-mon[36507]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-17T07:57:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:27 smithi053 ceph-mon[36437]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-17T07:57:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.827+0000 7f2305800700 1 -- 172.21.15.46:0/4042747028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2300108a40 msgr2=0x7f2300108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.827+0000 7f2305800700 1 --2- 172.21.15.46:0/4042747028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2300108a40 0x7f2300108e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009990 tx=0x7f22f0009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.828+0000 7f2305800700 1 -- 172.21.15.46:0/4042747028 shutdown_connections 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.828+0000 7f2305800700 1 --2- 172.21.15.46:0/4042747028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f230006b100 0x7f230010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.828+0000 7f2305800700 1 --2- 172.21.15.46:0/4042747028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2300108a40 0x7f2300108e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.828+0000 7f2305800700 1 --2- 172.21.15.46:0/4042747028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f230006a760 0x7f230006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.828+0000 7f2305800700 1 -- 172.21.15.46:0/4042747028 >> 172.21.15.46:0/4042747028 conn(0x7f2300078900 msgr2=0x7f2300078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.828+0000 7f2305800700 1 -- 172.21.15.46:0/4042747028 shutdown_connections 2024-04-17T07:57:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.828+0000 7f2305800700 1 -- 172.21.15.46:0/4042747028 wait complete. 2024-04-17T07:57:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.829+0000 7f2305800700 1 Processor -- start 2024-04-17T07:57:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.829+0000 7f2305800700 1 -- start start 2024-04-17T07:57:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f2305800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f230006a760 0x7f2300101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f2305800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230006b100 0x7f2300101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f2305800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2300108a40 0x7f23001024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f2305800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2300112d10 con 0x7f230006b100 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f2305800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2300112b90 con 0x7f2300108a40 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f2305800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2300112e90 con 0x7f230006a760 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f22f6000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230006b100 0x7f2300101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f22f6000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230006b100 0x7f2300101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57640/0 (socket says 172.21.15.46:57640) 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f22f6000700 1 -- 172.21.15.46:0/2959526780 learned_addr learned my addr 172.21.15.46:0/2959526780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.830+0000 7f22fea00700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f230006a760 0x7f2300101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.831+0000 7f22ff400700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2300108a40 0x7f23001024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.831+0000 7f22f6000700 1 -- 172.21.15.46:0/2959526780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f230006a760 msgr2=0x7f2300101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.831+0000 7f22f6000700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f230006a760 0x7f2300101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.831+0000 7f22f6000700 1 -- 172.21.15.46:0/2959526780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2300108a40 msgr2=0x7f23001024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.831+0000 7f22f6000700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2300108a40 0x7f23001024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.831+0000 7f22f6000700 1 -- 172.21.15.46:0/2959526780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23001a7770 con 0x7f230006b100 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.831+0000 7f22f6000700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230006b100 0x7f2300101f60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22e800e9a0 tx=0x7f22e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.832+0000 7f22f7e00700 1 -- 172.21.15.46:0/2959526780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e800c9c0 con 0x7f230006b100 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.832+0000 7f22f7e00700 1 -- 172.21.15.46:0/2959526780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e800cb60 con 0x7f230006b100 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.832+0000 7f22f7e00700 1 -- 172.21.15.46:0/2959526780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8018660 con 0x7f230006b100 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.832+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23001a7a60 con 0x7f230006b100 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.832+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23001a7ff0 con 0x7f230006b100 2024-04-17T07:57:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.833+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2300069740 con 0x7f230006b100 2024-04-17T07:57:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.835+0000 7f22f7e00700 1 -- 172.21.15.46:0/2959526780 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f22e800cd10 con 0x7f230006b100 2024-04-17T07:57:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.836+0000 7f22f7e00700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f22e40a3cc0 0x7f22e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.837+0000 7f22f7e00700 1 -- 172.21.15.46:0/2959526780 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f22e80d0920 con 0x7f230006b100 2024-04-17T07:57:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.837+0000 7f22fea00700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f22e40a3cc0 0x7f22e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.838+0000 7f22fea00700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f22e40a3cc0 0x7f22e40a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f22f00098e0 tx=0x7f22f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:27.841+0000 7f22f7e00700 1 -- 172.21.15.46:0/2959526780 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f22e809cf10 con 0x7f230006b100 2024-04-17T07:57:28.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.073+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2300062e90 con 0x7f22e40a3cc0 2024-04-17T07:57:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.084+0000 7f22f7e00700 1 -- 172.21.15.46:0/2959526780 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340605 (secure 0 0 0) 0x7f2300062e90 con 0x7f22e40a3cc0 2024-04-17T07:57:28.092 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.091+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f22e40a3cc0 msgr2=0x7f22e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.091+0000 7f2305800700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f22e40a3cc0 0x7f22e40a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f22f00098e0 tx=0x7f22f0005960 comp rx=0 tx=0).stop 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.091+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230006b100 msgr2=0x7f2300101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.091+0000 7f2305800700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230006b100 0x7f2300101f60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22e800e9a0 tx=0x7f22e800ecb0 comp rx=0 tx=0).stop 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 shutdown_connections 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f22e40a3cc0 0x7f22e40a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2300108a40 0x7f23001024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f230006a760 0x7f2300101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 --2- 172.21.15.46:0/2959526780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230006b100 0x7f2300101f60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 >> 172.21.15.46:0/2959526780 conn(0x7f2300078900 msgr2=0x7f23000ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:28.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 shutdown_connections 2024-04-17T07:57:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:28.092+0000 7f2305800700 1 -- 172.21.15.46:0/2959526780 wait complete. 2024-04-17T07:57:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:57:28.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:28 smithi046 ceph-mon[32038]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-17T07:57:28.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:28 smithi046 ceph-mon[32038]: Cluster is now healthy 2024-04-17T07:57:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:28 smithi053 ceph-mon[36437]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-17T07:57:28.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:28 smithi053 ceph-mon[36437]: Cluster is now healthy 2024-04-17T07:57:28.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:28 smithi062 ceph-mon[36507]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-17T07:57:28.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:28 smithi062 ceph-mon[36507]: Cluster is now healthy 2024-04-17T07:57:29.059 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-17T07:57:27.557077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658150,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":710,"num_read_kb":475,"num_write":357,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181973,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380336,"kb_used_data":14464,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561488,"statfs":{"total":1151940427776,"available":1151550963712,"internally_reserved":0,"allocated":14811136,"data_stored":8176082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":32,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002905"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986976+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:57:22.986976+0000","last_peered":"2024-04-17T07:57:22.986976+0000","last_clean":"2024-04-17T07:57:22.986976+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:57:22.986976+0000","last_undegraded":"2024-04-17T07:57:22.986976+0000","last_fullsized":"2024-04-17T07:57:22.986976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301099+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:57:23.301099+0000","last_peered":"2024-04-17T07:57:23.301099+0000","last_clean":"2024-04-17T07:57:23.301099+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:57:23.301099+0000","last_undegraded":"2024-04-17T07:57:23.301099+0000","last_fullsized":"2024-04-17T07:57:23.301099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300734+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:57:23.300734+0000","last_peered":"2024-04-17T07:57:23.300734+0000","last_clean":"2024-04-17T07:57:23.300734+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:57:23.300734+0000","last_undegraded":"2024-04-17T07:57:23.300734+0000","last_fullsized":"2024-04-17T07:57:23.300734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987715+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:57:22.987715+0000","last_peered":"2024-04-17T07:57:22.987715+0000","last_clean":"2024-04-17T07:57:22.987715+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:57:22.987715+0000","last_undegraded":"2024-04-17T07:57:22.987715+0000","last_fullsized":"2024-04-17T07:57:22.987715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989568+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:57:22.989568+0000","last_peered":"2024-04-17T07:57:22.989568+0000","last_clean":"2024-04-17T07:57:22.989568+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:57:22.989568+0000","last_undegraded":"2024-04-17T07:57:22.989568+0000","last_fullsized":"2024-04-17T07:57:22.989568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990211+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:57:22.990211+0000","last_peered":"2024-04-17T07:57:22.990211+0000","last_clean":"2024-04-17T07:57:22.990211+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:57:22.990211+0000","last_undegraded":"2024-04-17T07:57:22.990211+0000","last_fullsized":"2024-04-17T07:57:22.990211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989436+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:57:22.989436+0000","last_peered":"2024-04-17T07:57:22.989436+0000","last_clean":"2024-04-17T07:57:22.989436+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:57:22.989436+0000","last_undegraded":"2024-04-17T07:57:22.989436+0000","last_fullsized":"2024-04-17T07:57:22.989436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981303+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:57:22.981303+0000","last_peered":"2024-04-17T07:57:22.981303+0000","last_clean":"2024-04-17T07:57:22.981303+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:57:22.981303+0000","last_undegraded":"2024-04-17T07:57:22.981303+0000","last_fullsized":"2024-04-17T07:57:22.981303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298370+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:57:23.298370+0000","last_peered":"2024-04-17T07:57:23.298370+0000","last_clean":"2024-04-17T07:57:23.298370+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:57:23.298370+0000","last_undegraded":"2024-04-17T07:57:23.298370+0000","last_fullsized":"2024-04-17T07:57:23.298370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991013+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:57:22.991013+0000","last_peered":"2024-04-17T07:57:22.991013+0000","last_clean":"2024-04-17T07:57:22.991013+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:57:22.991013+0000","last_undegraded":"2024-04-17T07:57:22.991013+0000","last_fullsized":"2024-04-17T07:57:22.991013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990203+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:57:22.990203+0000","last_peered":"2024-04-17T07:57:22.990203+0000","last_clean":"2024-04-17T07:57:22.990203+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:57:22.990203+0000","last_undegraded":"2024-04-17T07:57:22.990203+0000","last_fullsized":"2024-04-17T07:57:22.990203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987774+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:57:22.987774+0000","last_peered":"2024-04-17T07:57:22.987774+0000","last_clean":"2024-04-17T07:57:22.987774+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:57:22.987774+0000","last_undegraded":"2024-04-17T07:57:22.987774+0000","last_fullsized":"2024-04-17T07:57:22.987774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984322+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:57:22.984322+0000","last_peered":"2024-04-17T07:57:22.984322+0000","last_clean":"2024-04-17T07:57:22.984322+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:57:22.984322+0000","last_undegraded":"2024-04-17T07:57:22.984322+0000","last_fullsized":"2024-04-17T07:57:22.984322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300112+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:57:23.300112+0000","last_peered":"2024-04-17T07:57:23.300112+0000","last_clean":"2024-04-17T07:57:23.300112+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:57:23.300112+0000","last_undegraded":"2024-04-17T07:57:23.300112+0000","last_fullsized":"2024-04-17T07:57:23.300112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300251+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:57:23.300251+0000","last_peered":"2024-04-17T07:57:23.300251+0000","last_clean":"2024-04-17T07:57:23.300251+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:57:23.300251+0000","last_undegraded":"2024-04-17T07:57:23.300251+0000","last_fullsized":"2024-04-17T07:57:23.300251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297603+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:57:23.297603+0000","last_peered":"2024-04-17T07:57:23.297603+0000","last_clean":"2024-04-17T07:57:23.297603+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:57:23.297603+0000","last_undegraded":"2024-04-17T07:57:23.297603+0000","last_fullsized":"2024-04-17T07:57:23.297603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986826+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:57:22.986826+0000","last_peered":"2024-04-17T07:57:22.986826+0000","last_clean":"2024-04-17T07:57:22.986826+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:57:22.986826+0000","last_undegraded":"2024-04-17T07:57:22.986826+0000","last_fullsized":"2024-04-17T07:57:22.986826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299034+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:57:23.299034+0000","last_peered":"2024-04-17T07:57:23.299034+0000","last_clean":"2024-04-17T07:57:23.299034+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:57:23.299034+0000","last_undegraded":"2024-04-17T07:57:23.299034+0000","last_fullsized":"2024-04-17T07:57:23.299034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990468+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:57:22.990468+0000","last_peered":"2024-04-17T07:57:22.990468+0000","last_clean":"2024-04-17T07:57:22.990468+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:57:22.990468+0000","last_undegraded":"2024-04-17T07:57:22.990468+0000","last_fullsized":"2024-04-17T07:57:22.990468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990092+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:57:22.990092+0000","last_peered":"2024-04-17T07:57:22.990092+0000","last_clean":"2024-04-17T07:57:22.990092+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:57:22.990092+0000","last_undegraded":"2024-04-17T07:57:22.990092+0000","last_fullsized":"2024-04-17T07:57:22.990092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299578+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:57:23.299578+0000","last_peered":"2024-04-17T07:57:23.299578+0000","last_clean":"2024-04-17T07:57:23.299578+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:57:23.299578+0000","last_undegraded":"2024-04-17T07:57:23.299578+0000","last_fullsized":"2024-04-17T07:57:23.299578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298902+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:57:23.298902+0000","last_peered":"2024-04-17T07:57:23.298902+0000","last_clean":"2024-04-17T07:57:23.298902+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:57:23.298902+0000","last_undegraded":"2024-04-17T07:57:23.298902+0000","last_fullsized":"2024-04-17T07:57:23.298902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987051+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:57:22.987051+0000","last_peered":"2024-04-17T07:57:22.987051+0000","last_clean":"2024-04-17T07:57:22.987051+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:57:22.987051+0000","last_undegraded":"2024-04-17T07:57:22.987051+0000","last_fullsized":"2024-04-17T07:57:22.987051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984474+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:57:22.984474+0000","last_peered":"2024-04-17T07:57:22.984474+0000","last_clean":"2024-04-17T07:57:22.984474+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:57:22.984474+0000","last_undegraded":"2024-04-17T07:57:22.984474+0000","last_fullsized":"2024-04-17T07:57:22.984474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301148+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:57:23.301148+0000","last_peered":"2024-04-17T07:57:23.301148+0000","last_clean":"2024-04-17T07:57:23.301148+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:57:23.301148+0000","last_undegraded":"2024-04-17T07:57:23.301148+0000","last_fullsized":"2024-04-17T07:57:23.301148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987157+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:57:22.987157+0000","last_peered":"2024-04-17T07:57:22.987157+0000","last_clean":"2024-04-17T07:57:22.987157+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:57:22.987157+0000","last_undegraded":"2024-04-17T07:57:22.987157+0000","last_fullsized":"2024-04-17T07:57:22.987157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.996476+0000","last_change":"2024-04-17T07:57:22.996476+0000","last_active":"2024-04-17T07:57:22.996476+0000","last_peered":"2024-04-17T07:57:22.996476+0000","last_clean":"2024-04-17T07:57:22.996476+0000","last_became_active":"2024-04-17T07:57:22.991654+0000","last_became_peered":"2024-04-17T07:57:22.991654+0000","last_unstale":"2024-04-17T07:57:22.996476+0000","last_undegraded":"2024-04-17T07:57:22.996476+0000","last_fullsized":"2024-04-17T07:57:22.996476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:33.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298000+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:57:23.298000+0000","last_peered":"2024-04-17T07:57:23.298000+0000","last_clean":"2024-04-17T07:57:23.298000+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:57:23.298000+0000","last_undegraded":"2024-04-17T07:57:23.298000+0000","last_fullsized":"2024-04-17T07:57:23.298000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987060+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:57:22.987060+0000","last_peered":"2024-04-17T07:57:22.987060+0000","last_clean":"2024-04-17T07:57:22.987060+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:57:22.987060+0000","last_undegraded":"2024-04-17T07:57:22.987060+0000","last_fullsized":"2024-04-17T07:57:22.987060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990006+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:57:22.990006+0000","last_peered":"2024-04-17T07:57:22.990006+0000","last_clean":"2024-04-17T07:57:22.990006+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:57:22.990006+0000","last_undegraded":"2024-04-17T07:57:22.990006+0000","last_fullsized":"2024-04-17T07:57:22.990006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981957+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:57:22.981957+0000","last_peered":"2024-04-17T07:57:22.981957+0000","last_clean":"2024-04-17T07:57:22.981957+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:57:22.981957+0000","last_undegraded":"2024-04-17T07:57:22.981957+0000","last_fullsized":"2024-04-17T07:57:22.981957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990345+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:57:22.990345+0000","last_peered":"2024-04-17T07:57:22.990345+0000","last_clean":"2024-04-17T07:57:22.990345+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:57:22.990345+0000","last_undegraded":"2024-04-17T07:57:22.990345+0000","last_fullsized":"2024-04-17T07:57:22.990345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":134,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:24.679877+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:57:24.679877+0000","last_peered":"2024-04-17T07:57:24.679877+0000","last_clean":"2024-04-17T07:57:24.679877+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:57:24.679877+0000","last_undegraded":"2024-04-17T07:57:24.679877+0000","last_fullsized":"2024-04-17T07:57:24.679877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989890+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:57:22.989890+0000","last_peered":"2024-04-17T07:57:22.989890+0000","last_clean":"2024-04-17T07:57:22.989890+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:57:22.989890+0000","last_undegraded":"2024-04-17T07:57:22.989890+0000","last_fullsized":"2024-04-17T07:57:22.989890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989695+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:57:22.989695+0000","last_peered":"2024-04-17T07:57:22.989695+0000","last_clean":"2024-04-17T07:57:22.989695+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:57:22.989695+0000","last_undegraded":"2024-04-17T07:57:22.989695+0000","last_fullsized":"2024-04-17T07:57:22.989695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990075+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:57:22.990075+0000","last_peered":"2024-04-17T07:57:22.990075+0000","last_clean":"2024-04-17T07:57:22.990075+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:57:22.990075+0000","last_undegraded":"2024-04-17T07:57:22.990075+0000","last_fullsized":"2024-04-17T07:57:22.990075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981716+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:57:22.981716+0000","last_peered":"2024-04-17T07:57:22.981716+0000","last_clean":"2024-04-17T07:57:22.981716+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:57:22.981716+0000","last_undegraded":"2024-04-17T07:57:22.981716+0000","last_fullsized":"2024-04-17T07:57:22.981716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986829+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:57:22.986829+0000","last_peered":"2024-04-17T07:57:22.986829+0000","last_clean":"2024-04-17T07:57:22.986829+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:57:22.986829+0000","last_undegraded":"2024-04-17T07:57:22.986829+0000","last_fullsized":"2024-04-17T07:57:22.986829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982797+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:57:22.982797+0000","last_peered":"2024-04-17T07:57:22.982797+0000","last_clean":"2024-04-17T07:57:22.982797+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:57:22.982797+0000","last_undegraded":"2024-04-17T07:57:22.982797+0000","last_fullsized":"2024-04-17T07:57:22.982797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.980936+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:57:22.980936+0000","last_peered":"2024-04-17T07:57:22.980936+0000","last_clean":"2024-04-17T07:57:22.980936+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:57:22.980936+0000","last_undegraded":"2024-04-17T07:57:22.980936+0000","last_fullsized":"2024-04-17T07:57:22.980936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300924+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:57:23.300924+0000","last_peered":"2024-04-17T07:57:23.300924+0000","last_clean":"2024-04-17T07:57:23.300924+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:57:23.300924+0000","last_undegraded":"2024-04-17T07:57:23.300924+0000","last_fullsized":"2024-04-17T07:57:23.300924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981394+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:57:22.981394+0000","last_peered":"2024-04-17T07:57:22.981394+0000","last_clean":"2024-04-17T07:57:22.981394+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:57:22.981394+0000","last_undegraded":"2024-04-17T07:57:22.981394+0000","last_fullsized":"2024-04-17T07:57:22.981394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299000+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:57:23.299000+0000","last_peered":"2024-04-17T07:57:23.299000+0000","last_clean":"2024-04-17T07:57:23.299000+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:57:23.299000+0000","last_undegraded":"2024-04-17T07:57:23.299000+0000","last_fullsized":"2024-04-17T07:57:23.299000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981197+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:57:22.981197+0000","last_peered":"2024-04-17T07:57:22.981197+0000","last_clean":"2024-04-17T07:57:22.981197+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:57:22.981197+0000","last_undegraded":"2024-04-17T07:57:22.981197+0000","last_fullsized":"2024-04-17T07:57:22.981197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":128,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:25.680630+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:57:25.680630+0000","last_peered":"2024-04-17T07:57:25.680630+0000","last_clean":"2024-04-17T07:57:25.680630+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:57:25.680630+0000","last_undegraded":"2024-04-17T07:57:25.680630+0000","last_fullsized":"2024-04-17T07:57:25.680630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298379+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:57:23.298379+0000","last_peered":"2024-04-17T07:57:23.298379+0000","last_clean":"2024-04-17T07:57:23.298379+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:57:23.298379+0000","last_undegraded":"2024-04-17T07:57:23.298379+0000","last_fullsized":"2024-04-17T07:57:23.298379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299142+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:57:23.299142+0000","last_peered":"2024-04-17T07:57:23.299142+0000","last_clean":"2024-04-17T07:57:23.299142+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:57:23.299142+0000","last_undegraded":"2024-04-17T07:57:23.299142+0000","last_fullsized":"2024-04-17T07:57:23.299142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989795+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:57:22.989795+0000","last_peered":"2024-04-17T07:57:22.989795+0000","last_clean":"2024-04-17T07:57:22.989795+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:57:22.989795+0000","last_undegraded":"2024-04-17T07:57:22.989795+0000","last_fullsized":"2024-04-17T07:57:22.989795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301097+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:57:23.301097+0000","last_peered":"2024-04-17T07:57:23.301097+0000","last_clean":"2024-04-17T07:57:23.301097+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:57:23.301097+0000","last_undegraded":"2024-04-17T07:57:23.301097+0000","last_fullsized":"2024-04-17T07:57:23.301097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987275+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:57:22.987275+0000","last_peered":"2024-04-17T07:57:22.987275+0000","last_clean":"2024-04-17T07:57:22.987275+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:57:22.987275+0000","last_undegraded":"2024-04-17T07:57:22.987275+0000","last_fullsized":"2024-04-17T07:57:22.987275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981928+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:57:22.981928+0000","last_peered":"2024-04-17T07:57:22.981928+0000","last_clean":"2024-04-17T07:57:22.981928+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:57:22.981928+0000","last_undegraded":"2024-04-17T07:57:22.981928+0000","last_fullsized":"2024-04-17T07:57:22.981928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298113+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:57:23.298113+0000","last_peered":"2024-04-17T07:57:23.298113+0000","last_clean":"2024-04-17T07:57:23.298113+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:57:23.298113+0000","last_undegraded":"2024-04-17T07:57:23.298113+0000","last_fullsized":"2024-04-17T07:57:23.298113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298615+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:57:23.298615+0000","last_peered":"2024-04-17T07:57:23.298615+0000","last_clean":"2024-04-17T07:57:23.298615+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:57:23.298615+0000","last_undegraded":"2024-04-17T07:57:23.298615+0000","last_fullsized":"2024-04-17T07:57:23.298615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":261,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:26.681978+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:57:26.681978+0000","last_peered":"2024-04-17T07:57:26.681978+0000","last_clean":"2024-04-17T07:57:26.681978+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:57:26.681978+0000","last_undegraded":"2024-04-17T07:57:26.681978+0000","last_fullsized":"2024-04-17T07:57:26.681978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":195,"num_read_kb":160,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991075+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:57:22.991075+0000","last_peered":"2024-04-17T07:57:22.991075+0000","last_clean":"2024-04-17T07:57:22.991075+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:57:22.991075+0000","last_undegraded":"2024-04-17T07:57:22.991075+0000","last_fullsized":"2024-04-17T07:57:22.991075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982168+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:57:22.982168+0000","last_peered":"2024-04-17T07:57:22.982168+0000","last_clean":"2024-04-17T07:57:22.982168+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:57:22.982168+0000","last_undegraded":"2024-04-17T07:57:22.982168+0000","last_fullsized":"2024-04-17T07:57:22.982168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989392+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:57:22.989392+0000","last_peered":"2024-04-17T07:57:22.989392+0000","last_clean":"2024-04-17T07:57:22.989392+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:57:22.989392+0000","last_undegraded":"2024-04-17T07:57:22.989392+0000","last_fullsized":"2024-04-17T07:57:22.989392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983358+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:57:22.983358+0000","last_peered":"2024-04-17T07:57:22.983358+0000","last_clean":"2024-04-17T07:57:22.983358+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:57:22.983358+0000","last_undegraded":"2024-04-17T07:57:22.983358+0000","last_fullsized":"2024-04-17T07:57:22.983358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986309+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:57:22.986309+0000","last_peered":"2024-04-17T07:57:22.986309+0000","last_clean":"2024-04-17T07:57:22.986309+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:57:22.986309+0000","last_undegraded":"2024-04-17T07:57:22.986309+0000","last_fullsized":"2024-04-17T07:57:22.986309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981818+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:57:22.981818+0000","last_peered":"2024-04-17T07:57:22.981818+0000","last_clean":"2024-04-17T07:57:22.981818+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:57:22.981818+0000","last_undegraded":"2024-04-17T07:57:22.981818+0000","last_fullsized":"2024-04-17T07:57:22.981818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989572+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:57:22.989572+0000","last_peered":"2024-04-17T07:57:22.989572+0000","last_clean":"2024-04-17T07:57:22.989572+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:57:22.989572+0000","last_undegraded":"2024-04-17T07:57:22.989572+0000","last_fullsized":"2024-04-17T07:57:22.989572+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299360+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:57:23.299360+0000","last_peered":"2024-04-17T07:57:23.299360+0000","last_clean":"2024-04-17T07:57:23.299360+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:57:23.299360+0000","last_undegraded":"2024-04-17T07:57:23.299360+0000","last_fullsized":"2024-04-17T07:57:23.299360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:24.998402+0000","last_change":"2024-04-17T07:57:22.995880+0000","last_active":"2024-04-17T07:57:24.998402+0000","last_peered":"2024-04-17T07:57:24.998402+0000","last_clean":"2024-04-17T07:57:24.998402+0000","last_became_active":"2024-04-17T07:57:22.991967+0000","last_became_peered":"2024-04-17T07:57:22.991967+0000","last_unstale":"2024-04-17T07:57:24.998402+0000","last_undegraded":"2024-04-17T07:57:24.998402+0000","last_fullsized":"2024-04-17T07:57:24.998402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:27.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989483+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:57:22.989483+0000","last_peered":"2024-04-17T07:57:22.989483+0000","last_clean":"2024-04-17T07:57:22.989483+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:57:22.989483+0000","last_undegraded":"2024-04-17T07:57:22.989483+0000","last_fullsized":"2024-04-17T07:57:22.989483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984496+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:57:22.984496+0000","last_peered":"2024-04-17T07:57:22.984496+0000","last_clean":"2024-04-17T07:57:22.984496+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:57:22.984496+0000","last_undegraded":"2024-04-17T07:57:22.984496+0000","last_fullsized":"2024-04-17T07:57:22.984496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989772+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:57:22.989772+0000","last_peered":"2024-04-17T07:57:22.989772+0000","last_clean":"2024-04-17T07:57:22.989772+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:57:22.989772+0000","last_undegraded":"2024-04-17T07:57:22.989772+0000","last_fullsized":"2024-04-17T07:57:22.989772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981395+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:57:22.981395+0000","last_peered":"2024-04-17T07:57:22.981395+0000","last_clean":"2024-04-17T07:57:22.981395+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:57:22.981395+0000","last_undegraded":"2024-04-17T07:57:22.981395+0000","last_fullsized":"2024-04-17T07:57:22.981395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981314+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:57:22.981314+0000","last_peered":"2024-04-17T07:57:22.981314+0000","last_clean":"2024-04-17T07:57:22.981314+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:57:22.981314+0000","last_undegraded":"2024-04-17T07:57:22.981314+0000","last_fullsized":"2024-04-17T07:57:22.981314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991206+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:57:22.991206+0000","last_peered":"2024-04-17T07:57:22.991206+0000","last_clean":"2024-04-17T07:57:22.991206+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:57:22.991206+0000","last_undegraded":"2024-04-17T07:57:22.991206+0000","last_fullsized":"2024-04-17T07:57:22.991206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986817+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:57:22.986817+0000","last_peered":"2024-04-17T07:57:22.986817+0000","last_clean":"2024-04-17T07:57:22.986817+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:57:22.986817+0000","last_undegraded":"2024-04-17T07:57:22.986817+0000","last_fullsized":"2024-04-17T07:57:22.986817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989642+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:57:22.989642+0000","last_peered":"2024-04-17T07:57:22.989642+0000","last_clean":"2024-04-17T07:57:22.989642+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:57:22.989642+0000","last_undegraded":"2024-04-17T07:57:22.989642+0000","last_fullsized":"2024-04-17T07:57:22.989642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299560+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:57:23.299560+0000","last_peered":"2024-04-17T07:57:23.299560+0000","last_clean":"2024-04-17T07:57:23.299560+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:57:23.299560+0000","last_undegraded":"2024-04-17T07:57:23.299560+0000","last_fullsized":"2024-04-17T07:57:23.299560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990960+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:57:22.990960+0000","last_peered":"2024-04-17T07:57:22.990960+0000","last_clean":"2024-04-17T07:57:22.990960+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:57:22.990960+0000","last_undegraded":"2024-04-17T07:57:22.990960+0000","last_fullsized":"2024-04-17T07:57:22.990960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982460+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:57:22.982460+0000","last_peered":"2024-04-17T07:57:22.982460+0000","last_clean":"2024-04-17T07:57:22.982460+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:57:22.982460+0000","last_undegraded":"2024-04-17T07:57:22.982460+0000","last_fullsized":"2024-04-17T07:57:22.982460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989350+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:57:22.989350+0000","last_peered":"2024-04-17T07:57:22.989350+0000","last_clean":"2024-04-17T07:57:22.989350+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:57:22.989350+0000","last_undegraded":"2024-04-17T07:57:22.989350+0000","last_fullsized":"2024-04-17T07:57:22.989350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990505+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:57:22.990505+0000","last_peered":"2024-04-17T07:57:22.990505+0000","last_clean":"2024-04-17T07:57:22.990505+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:57:22.990505+0000","last_undegraded":"2024-04-17T07:57:22.990505+0000","last_fullsized":"2024-04-17T07:57:22.990505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.985641+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:57:22.985641+0000","last_peered":"2024-04-17T07:57:22.985641+0000","last_clean":"2024-04-17T07:57:22.985641+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:57:22.985641+0000","last_undegraded":"2024-04-17T07:57:22.985641+0000","last_fullsized":"2024-04-17T07:57:22.985641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":128,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:24.997630+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:57:24.997630+0000","last_peered":"2024-04-17T07:57:24.997630+0000","last_clean":"2024-04-17T07:57:24.997630+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:57:24.997630+0000","last_undegraded":"2024-04-17T07:57:24.997630+0000","last_fullsized":"2024-04-17T07:57:24.997630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989145+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:57:22.989145+0000","last_peered":"2024-04-17T07:57:22.989145+0000","last_clean":"2024-04-17T07:57:22.989145+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:57:22.989145+0000","last_undegraded":"2024-04-17T07:57:22.989145+0000","last_fullsized":"2024-04-17T07:57:22.989145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982533+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:57:22.982533+0000","last_peered":"2024-04-17T07:57:22.982533+0000","last_clean":"2024-04-17T07:57:22.982533+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:57:22.982533+0000","last_undegraded":"2024-04-17T07:57:22.982533+0000","last_fullsized":"2024-04-17T07:57:22.982533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987240+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:57:22.987240+0000","last_peered":"2024-04-17T07:57:22.987240+0000","last_clean":"2024-04-17T07:57:22.987240+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:57:22.987240+0000","last_undegraded":"2024-04-17T07:57:22.987240+0000","last_fullsized":"2024-04-17T07:57:22.987240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989188+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:57:22.989188+0000","last_peered":"2024-04-17T07:57:22.989188+0000","last_clean":"2024-04-17T07:57:22.989188+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:57:22.989188+0000","last_undegraded":"2024-04-17T07:57:22.989188+0000","last_fullsized":"2024-04-17T07:57:22.989188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982007+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:57:22.982007+0000","last_peered":"2024-04-17T07:57:22.982007+0000","last_clean":"2024-04-17T07:57:22.982007+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:57:22.982007+0000","last_undegraded":"2024-04-17T07:57:22.982007+0000","last_fullsized":"2024-04-17T07:57:22.982007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982286+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:57:22.982286+0000","last_peered":"2024-04-17T07:57:22.982286+0000","last_clean":"2024-04-17T07:57:22.982286+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:57:22.982286+0000","last_undegraded":"2024-04-17T07:57:22.982286+0000","last_fullsized":"2024-04-17T07:57:22.982286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989408+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:57:22.989408+0000","last_peered":"2024-04-17T07:57:22.989408+0000","last_clean":"2024-04-17T07:57:22.989408+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:57:22.989408+0000","last_undegraded":"2024-04-17T07:57:22.989408+0000","last_fullsized":"2024-04-17T07:57:22.989408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986287+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:57:22.986287+0000","last_peered":"2024-04-17T07:57:22.986287+0000","last_clean":"2024-04-17T07:57:22.986287+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:57:22.986287+0000","last_undegraded":"2024-04-17T07:57:22.986287+0000","last_fullsized":"2024-04-17T07:57:22.986287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990959+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:57:22.990959+0000","last_peered":"2024-04-17T07:57:22.990959+0000","last_clean":"2024-04-17T07:57:22.990959+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:57:22.990959+0000","last_undegraded":"2024-04-17T07:57:22.990959+0000","last_fullsized":"2024-04-17T07:57:22.990959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983324+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:57:22.983324+0000","last_peered":"2024-04-17T07:57:22.983324+0000","last_clean":"2024-04-17T07:57:22.983324+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:57:22.983324+0000","last_undegraded":"2024-04-17T07:57:22.983324+0000","last_fullsized":"2024-04-17T07:57:22.983324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.980943+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:57:22.980943+0000","last_peered":"2024-04-17T07:57:22.980943+0000","last_clean":"2024-04-17T07:57:22.980943+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:57:22.980943+0000","last_undegraded":"2024-04-17T07:57:22.980943+0000","last_fullsized":"2024-04-17T07:57:22.980943+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299165+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:57:23.299165+0000","last_peered":"2024-04-17T07:57:23.299165+0000","last_clean":"2024-04-17T07:57:23.299165+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:57:23.299165+0000","last_undegraded":"2024-04-17T07:57:23.299165+0000","last_fullsized":"2024-04-17T07:57:23.299165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984112+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:57:22.984112+0000","last_peered":"2024-04-17T07:57:22.984112+0000","last_clean":"2024-04-17T07:57:22.984112+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:57:22.984112+0000","last_undegraded":"2024-04-17T07:57:22.984112+0000","last_fullsized":"2024-04-17T07:57:22.984112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989763+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:57:22.989763+0000","last_peered":"2024-04-17T07:57:22.989763+0000","last_clean":"2024-04-17T07:57:22.989763+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:57:22.989763+0000","last_undegraded":"2024-04-17T07:57:22.989763+0000","last_fullsized":"2024-04-17T07:57:22.989763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987850+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:57:22.987850+0000","last_peered":"2024-04-17T07:57:22.987850+0000","last_clean":"2024-04-17T07:57:22.987850+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:57:22.987850+0000","last_undegraded":"2024-04-17T07:57:22.987850+0000","last_fullsized":"2024-04-17T07:57:22.987850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301271+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:57:23.301271+0000","last_peered":"2024-04-17T07:57:23.301271+0000","last_clean":"2024-04-17T07:57:23.301271+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:57:23.301271+0000","last_undegraded":"2024-04-17T07:57:23.301271+0000","last_fullsized":"2024-04-17T07:57:23.301271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987589+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:57:22.987589+0000","last_peered":"2024-04-17T07:57:22.987589+0000","last_clean":"2024-04-17T07:57:22.987589+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:57:22.987589+0000","last_undegraded":"2024-04-17T07:57:22.987589+0000","last_fullsized":"2024-04-17T07:57:22.987589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297678+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:57:23.297678+0000","last_peered":"2024-04-17T07:57:23.297678+0000","last_clean":"2024-04-17T07:57:23.297678+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:57:23.297678+0000","last_undegraded":"2024-04-17T07:57:23.297678+0000","last_fullsized":"2024-04-17T07:57:23.297678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989036+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:57:22.989036+0000","last_peered":"2024-04-17T07:57:22.989036+0000","last_clean":"2024-04-17T07:57:22.989036+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:57:22.989036+0000","last_undegraded":"2024-04-17T07:57:22.989036+0000","last_fullsized":"2024-04-17T07:57:22.989036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299526+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:57:23.299526+0000","last_peered":"2024-04-17T07:57:23.299526+0000","last_clean":"2024-04-17T07:57:23.299526+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:57:23.299526+0000","last_undegraded":"2024-04-17T07:57:23.299526+0000","last_fullsized":"2024-04-17T07:57:23.299526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981526+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:57:22.981526+0000","last_peered":"2024-04-17T07:57:22.981526+0000","last_clean":"2024-04-17T07:57:22.981526+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:57:22.981526+0000","last_undegraded":"2024-04-17T07:57:22.981526+0000","last_fullsized":"2024-04-17T07:57:22.981526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986228+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:57:22.986228+0000","last_peered":"2024-04-17T07:57:22.986228+0000","last_clean":"2024-04-17T07:57:22.986228+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:57:22.986228+0000","last_undegraded":"2024-04-17T07:57:22.986228+0000","last_fullsized":"2024-04-17T07:57:22.986228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987327+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:57:22.987327+0000","last_peered":"2024-04-17T07:57:22.987327+0000","last_clean":"2024-04-17T07:57:22.987327+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:57:22.987327+0000","last_undegraded":"2024-04-17T07:57:22.987327+0000","last_fullsized":"2024-04-17T07:57:22.987327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990287+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:57:22.990287+0000","last_peered":"2024-04-17T07:57:22.990287+0000","last_clean":"2024-04-17T07:57:22.990287+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:57:22.990287+0000","last_undegraded":"2024-04-17T07:57:22.990287+0000","last_fullsized":"2024-04-17T07:57:22.990287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983499+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:57:22.983499+0000","last_peered":"2024-04-17T07:57:22.983499+0000","last_clean":"2024-04-17T07:57:22.983499+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:57:22.983499+0000","last_undegraded":"2024-04-17T07:57:22.983499+0000","last_fullsized":"2024-04-17T07:57:22.983499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987653+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:57:22.987653+0000","last_peered":"2024-04-17T07:57:22.987653+0000","last_clean":"2024-04-17T07:57:22.987653+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:57:22.987653+0000","last_undegraded":"2024-04-17T07:57:22.987653+0000","last_fullsized":"2024-04-17T07:57:22.987653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989692+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:57:22.989692+0000","last_peered":"2024-04-17T07:57:22.989692+0000","last_clean":"2024-04-17T07:57:22.989692+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:57:22.989692+0000","last_undegraded":"2024-04-17T07:57:22.989692+0000","last_fullsized":"2024-04-17T07:57:22.989692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989471+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:57:22.989471+0000","last_peered":"2024-04-17T07:57:22.989471+0000","last_clean":"2024-04-17T07:57:22.989471+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:57:22.989471+0000","last_undegraded":"2024-04-17T07:57:22.989471+0000","last_fullsized":"2024-04-17T07:57:22.989471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297723+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:57:23.297723+0000","last_peered":"2024-04-17T07:57:23.297723+0000","last_clean":"2024-04-17T07:57:23.297723+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:57:23.297723+0000","last_undegraded":"2024-04-17T07:57:23.297723+0000","last_fullsized":"2024-04-17T07:57:23.297723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989968+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:57:22.989968+0000","last_peered":"2024-04-17T07:57:22.989968+0000","last_clean":"2024-04-17T07:57:22.989968+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:57:22.989968+0000","last_undegraded":"2024-04-17T07:57:22.989968+0000","last_fullsized":"2024-04-17T07:57:22.989968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990328+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:57:22.990328+0000","last_peered":"2024-04-17T07:57:22.990328+0000","last_clean":"2024-04-17T07:57:22.990328+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:57:22.990328+0000","last_undegraded":"2024-04-17T07:57:22.990328+0000","last_fullsized":"2024-04-17T07:57:22.990328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299788+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:57:23.299788+0000","last_peered":"2024-04-17T07:57:23.299788+0000","last_clean":"2024-04-17T07:57:23.299788+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:57:23.299788+0000","last_undegraded":"2024-04-17T07:57:23.299788+0000","last_fullsized":"2024-04-17T07:57:23.299788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982080+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:57:22.982080+0000","last_peered":"2024-04-17T07:57:22.982080+0000","last_clean":"2024-04-17T07:57:22.982080+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:57:22.982080+0000","last_undegraded":"2024-04-17T07:57:22.982080+0000","last_fullsized":"2024-04-17T07:57:22.982080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989766+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:57:22.989766+0000","last_peered":"2024-04-17T07:57:22.989766+0000","last_clean":"2024-04-17T07:57:22.989766+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:57:22.989766+0000","last_undegraded":"2024-04-17T07:57:22.989766+0000","last_fullsized":"2024-04-17T07:57:22.989766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981178+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:57:22.981178+0000","last_peered":"2024-04-17T07:57:22.981178+0000","last_clean":"2024-04-17T07:57:22.981178+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:57:22.981178+0000","last_undegraded":"2024-04-17T07:57:22.981178+0000","last_fullsized":"2024-04-17T07:57:22.981178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984793+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:57:22.984793+0000","last_peered":"2024-04-17T07:57:22.984793+0000","last_clean":"2024-04-17T07:57:22.984793+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:57:22.984793+0000","last_undegraded":"2024-04-17T07:57:22.984793+0000","last_fullsized":"2024-04-17T07:57:22.984793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989884+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:57:22.989884+0000","last_peered":"2024-04-17T07:57:22.989884+0000","last_clean":"2024-04-17T07:57:22.989884+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:57:22.989884+0000","last_undegraded":"2024-04-17T07:57:22.989884+0000","last_fullsized":"2024-04-17T07:57:22.989884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986441+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:57:22.986441+0000","last_peered":"2024-04-17T07:57:22.986441+0000","last_clean":"2024-04-17T07:57:22.986441+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:57:22.986441+0000","last_undegraded":"2024-04-17T07:57:22.986441+0000","last_fullsized":"2024-04-17T07:57:22.986441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990047+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:57:22.990047+0000","last_peered":"2024-04-17T07:57:22.990047+0000","last_clean":"2024-04-17T07:57:22.990047+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:57:22.990047+0000","last_undegraded":"2024-04-17T07:57:22.990047+0000","last_fullsized":"2024-04-17T07:57:22.990047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990157+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:57:22.990157+0000","last_peered":"2024-04-17T07:57:22.990157+0000","last_clean":"2024-04-17T07:57:22.990157+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:57:22.990157+0000","last_undegraded":"2024-04-17T07:57:22.990157+0000","last_fullsized":"2024-04-17T07:57:22.990157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300310+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:57:23.300310+0000","last_peered":"2024-04-17T07:57:23.300310+0000","last_clean":"2024-04-17T07:57:23.300310+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:57:23.300310+0000","last_undegraded":"2024-04-17T07:57:23.300310+0000","last_fullsized":"2024-04-17T07:57:23.300310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983015+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:57:22.983015+0000","last_peered":"2024-04-17T07:57:22.983015+0000","last_clean":"2024-04-17T07:57:22.983015+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:57:22.983015+0000","last_undegraded":"2024-04-17T07:57:22.983015+0000","last_fullsized":"2024-04-17T07:57:22.983015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986662+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:57:22.986662+0000","last_peered":"2024-04-17T07:57:22.986662+0000","last_clean":"2024-04-17T07:57:22.986662+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:57:22.986662+0000","last_undegraded":"2024-04-17T07:57:22.986662+0000","last_fullsized":"2024-04-17T07:57:22.986662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990470+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:57:22.990470+0000","last_peered":"2024-04-17T07:57:22.990470+0000","last_clean":"2024-04-17T07:57:22.990470+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:57:22.990470+0000","last_undegraded":"2024-04-17T07:57:22.990470+0000","last_fullsized":"2024-04-17T07:57:22.990470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984574+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:57:22.984574+0000","last_peered":"2024-04-17T07:57:22.984574+0000","last_clean":"2024-04-17T07:57:22.984574+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:57:22.984574+0000","last_undegraded":"2024-04-17T07:57:22.984574+0000","last_fullsized":"2024-04-17T07:57:22.984574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981579+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:57:22.981579+0000","last_peered":"2024-04-17T07:57:22.981579+0000","last_clean":"2024-04-17T07:57:22.981579+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:57:22.981579+0000","last_undegraded":"2024-04-17T07:57:22.981579+0000","last_fullsized":"2024-04-17T07:57:22.981579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981814+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:57:22.981814+0000","last_peered":"2024-04-17T07:57:22.981814+0000","last_clean":"2024-04-17T07:57:22.981814+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:57:22.981814+0000","last_undegraded":"2024-04-17T07:57:22.981814+0000","last_fullsized":"2024-04-17T07:57:22.981814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297986+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:57:23.297986+0000","last_peered":"2024-04-17T07:57:23.297986+0000","last_clean":"2024-04-17T07:57:23.297986+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:57:23.297986+0000","last_undegraded":"2024-04-17T07:57:23.297986+0000","last_fullsized":"2024-04-17T07:57:23.297986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984520+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:57:22.984520+0000","last_peered":"2024-04-17T07:57:22.984520+0000","last_clean":"2024-04-17T07:57:22.984520+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:57:22.984520+0000","last_undegraded":"2024-04-17T07:57:22.984520+0000","last_fullsized":"2024-04-17T07:57:22.984520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986489+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:57:22.986489+0000","last_peered":"2024-04-17T07:57:22.986489+0000","last_clean":"2024-04-17T07:57:22.986489+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:57:22.986489+0000","last_undegraded":"2024-04-17T07:57:22.986489+0000","last_fullsized":"2024-04-17T07:57:22.986489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67548,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":584,"num_read_kb":366,"num_write":174,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1748992,"data_stored":1170263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":523921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070519,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":514475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1115207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":577785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":579667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1107287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:57:29.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph status --format=json 2024-04-17T07:57:29.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:29 smithi046 ceph-mon[32038]: pgmap v386: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 8 op/s; 139 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:29.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:29 smithi046 ceph-mon[32038]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:29 smithi053 ceph-mon[36437]: pgmap v386: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 8 op/s; 139 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:29 smithi053 ceph-mon[36437]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:29 smithi062 ceph-mon[36507]: pgmap v386: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 8 op/s; 139 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:29 smithi062 ceph-mon[36507]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:31.304 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:31 smithi046 ceph-mon[32038]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 6 op/s; 139 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:31 smithi053 ceph-mon[36437]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 6 op/s; 139 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:31.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:31 smithi062 ceph-mon[36507]: pgmap v387: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 6 op/s; 139 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 -- 172.21.15.46:0/4069478173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc104de0 msgr2=0x7f4acc1051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 --2- 172.21.15.46:0/4069478173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc104de0 0x7f4acc1051c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00099c0 tx=0x7f4ac0009cd0 comp rx=0 tx=0).stop 2024-04-17T07:57:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 -- 172.21.15.46:0/4069478173 shutdown_connections 2024-04-17T07:57:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 --2- 172.21.15.46:0/4069478173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc109ec0 0x7f4acc111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 --2- 172.21.15.46:0/4069478173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc104de0 0x7f4acc1051c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 --2- 172.21.15.46:0/4069478173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4acc105790 0x7f4acc1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 -- 172.21.15.46:0/4069478173 >> 172.21.15.46:0/4069478173 conn(0x7f4acc100670 msgr2=0x7f4acc102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.861+0000 7f4ad2600700 1 -- 172.21.15.46:0/4069478173 shutdown_connections 2024-04-17T07:57:31.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.862+0000 7f4ad2600700 1 -- 172.21.15.46:0/4069478173 wait complete. 2024-04-17T07:57:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.862+0000 7f4ad2600700 1 Processor -- start 2024-04-17T07:57:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad2600700 1 -- start start 2024-04-17T07:57:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad2600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc104de0 0x7f4acc19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad2600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4acc105790 0x7f4acc19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad2600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc109ec0 0x7f4acc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad2600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc114f00 con 0x7f4acc109ec0 2024-04-17T07:57:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad2600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc114d80 con 0x7f4acc104de0 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad2600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc115080 con 0x7f4acc105790 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.863+0000 7f4ad1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc109ec0 0x7f4acc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc109ec0 0x7f4acc19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57660/0 (socket says 172.21.15.46:57660) 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4acbe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4acc105790 0x7f4acc19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1c00700 1 -- 172.21.15.46:0/3835583852 learned_addr learned my addr 172.21.15.46:0/3835583852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1200700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc104de0 0x7f4acc19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1200700 1 -- 172.21.15.46:0/3835583852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4acc105790 msgr2=0x7f4acc19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1200700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4acc105790 0x7f4acc19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1200700 1 -- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc109ec0 msgr2=0x7f4acc19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1200700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc109ec0 0x7f4acc19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1200700 1 -- 172.21.15.46:0/3835583852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc1a0340 con 0x7f4acc104de0 2024-04-17T07:57:31.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1c00700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc109ec0 0x7f4acc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.864+0000 7f4ad1200700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc104de0 0x7f4acc19b330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00098e0 tx=0x7f4ac0003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.865+0000 7f4ac9600700 1 -- 172.21.15.46:0/3835583852 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0009e60 con 0x7f4acc104de0 2024-04-17T07:57:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.865+0000 7f4ac9600700 1 -- 172.21.15.46:0/3835583852 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0003680 con 0x7f4acc104de0 2024-04-17T07:57:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.865+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc1a05d0 con 0x7f4acc104de0 2024-04-17T07:57:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.865+0000 7f4ac9600700 1 -- 172.21.15.46:0/3835583852 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000fd40 con 0x7f4acc104de0 2024-04-17T07:57:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.865+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a7eb0 con 0x7f4acc104de0 2024-04-17T07:57:31.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.866+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc1062c0 con 0x7f4acc104de0 2024-04-17T07:57:31.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.868+0000 7f4ac9600700 1 -- 172.21.15.46:0/3835583852 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ac000f460 con 0x7f4acc104de0 2024-04-17T07:57:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.869+0000 7f4ac9600700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4ab40a3c40 0x7f4ab40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.870+0000 7f4ac9600700 1 -- 172.21.15.46:0/3835583852 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4ac00d1760 con 0x7f4acc104de0 2024-04-17T07:57:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.870+0000 7f4acbe00700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4ab40a3c40 0x7f4ab40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:31.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.871+0000 7f4acbe00700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4ab40a3c40 0x7f4ab40a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4acc10b470 tx=0x7f4ab80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:31.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:31.874+0000 7f4ac9600700 1 -- 172.21.15.46:0/3835583852 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ac0013070 con 0x7f4acc104de0 2024-04-17T07:57:32.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.197+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4acc19c690 con 0x7f4acc104de0 2024-04-17T07:57:32.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.197+0000 7f4ac9600700 1 -- 172.21.15.46:0/3835583852 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1662 (secure 0 0 0) 0x7f4ac009dde0 con 0x7f4acc104de0 2024-04-17T07:57:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:32.198 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":540,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":12,"num_up_osds":12,"osd_up_since":1713340466,"num_in_osds":12,"osd_in_since":1713340446,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":658150,"bytes_used":389480448,"bytes_avail":1151550947328,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":116,"recovering_keys_per_sec":1,"num_objects_recovered":4,"num_bytes_recovered":1120,"num_keys_recovered":10,"read_bytes_sec":2243,"write_bytes_sec":320,"read_op_per_sec":4,"write_op_per_sec":0},"fsmap":{"epoch":23,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24505},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24517},{"filesystem_id":1,"rank":0,"name":"i","status":"up:standby-replay","gid":24538},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24542},{"filesystem_id":1,"rank":1,"name":"g","status":"up:active","gid":24556},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24596}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.46:8443/"}},"servicemap":{"epoch":27,"modified":"2024-04-17T07:56:59.544752+0000","services":{}},"progress_events":{}} 2024-04-17T07:57:32.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4ab40a3c40 msgr2=0x7f4ab40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4ab40a3c40 0x7f4ab40a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4acc10b470 tx=0x7f4ab80073d0 comp rx=0 tx=0).stop 2024-04-17T07:57:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc104de0 msgr2=0x7f4acc19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc104de0 0x7f4acc19b330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00098e0 tx=0x7f4ac0003e00 comp rx=0 tx=0).stop 2024-04-17T07:57:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 shutdown_connections 2024-04-17T07:57:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4ab40a3c40 0x7f4ab40a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4acc109ec0 0x7f4acc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4acc104de0 0x7f4acc19b330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 --2- 172.21.15.46:0/3835583852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4acc105790 0x7f4acc19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.200+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 >> 172.21.15.46:0/3835583852 conn(0x7f4acc100670 msgr2=0x7f4acc102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.201+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 shutdown_connections 2024-04-17T07:57:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:32.201+0000 7f4ad2600700 1 -- 172.21.15.46:0/3835583852 wait complete. 2024-04-17T07:57:33.063 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-04-17T07:57:33.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T07:57:33.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:33 smithi046 ceph-mon[32038]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 320 B/s wr, 5 op/s; 116 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:33.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:33 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3835583852' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-17T07:57:33.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:33 smithi053 ceph-mon[36437]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 320 B/s wr, 5 op/s; 116 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:33.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:33 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3835583852' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-17T07:57:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:33.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:33 smithi062 ceph-mon[36507]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 320 B/s wr, 5 op/s; 116 B/s, 1 keys/s, 0 objects/s recovering 2024-04-17T07:57:33.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:33 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3835583852' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-17T07:57:35.234 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:35 smithi046 ceph-mon[32038]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 290 B/s wr, 6 op/s; 105 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:35.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:35 smithi053 ceph-mon[36437]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 290 B/s wr, 6 op/s; 105 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:35.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:35 smithi062 ceph-mon[36507]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 290 B/s wr, 6 op/s; 105 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:35.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.762+0000 7efc1ee00700 1 -- 172.21.15.46:0/4201628843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc181092e0 msgr2=0x7efc181096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.762+0000 7efc1ee00700 1 --2- 172.21.15.46:0/4201628843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc181092e0 0x7efc181096c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc080099c0 tx=0x7efc08009cd0 comp rx=0 tx=0).stop 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.762+0000 7efc1ee00700 1 -- 172.21.15.46:0/4201628843 shutdown_connections 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.762+0000 7efc1ee00700 1 --2- 172.21.15.46:0/4201628843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc1810e3c0 0x7efc18115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.762+0000 7efc1ee00700 1 --2- 172.21.15.46:0/4201628843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc181092e0 0x7efc181096c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.762+0000 7efc1ee00700 1 --2- 172.21.15.46:0/4201628843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc18109c90 0x7efc1810dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.763+0000 7efc1ee00700 1 -- 172.21.15.46:0/4201628843 >> 172.21.15.46:0/4201628843 conn(0x7efc18076220 msgr2=0x7efc18107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.763+0000 7efc1ee00700 1 -- 172.21.15.46:0/4201628843 shutdown_connections 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.763+0000 7efc1ee00700 1 -- 172.21.15.46:0/4201628843 wait complete. 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 Processor -- start 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 -- start start 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc181092e0 0x7efc18082f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc18109c90 0x7efc180834c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:35.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1810e3c0 0x7efc1807c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:35.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc181193a0 con 0x7efc1810e3c0 2024-04-17T07:57:35.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc18119220 con 0x7efc18109c90 2024-04-17T07:57:35.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.764+0000 7efc1ee00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc18119520 con 0x7efc181092e0 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.765+0000 7efc17e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc181092e0 0x7efc18082f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.765+0000 7efc17e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc181092e0 0x7efc18082f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40418/0 (socket says 172.21.15.46:40418) 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.765+0000 7efc17e00700 1 -- 172.21.15.46:0/3418140743 learned_addr learned my addr 172.21.15.46:0/3418140743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.765+0000 7efc1ce00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1810e3c0 0x7efc1807c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.765+0000 7efc17400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc18109c90 0x7efc180834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.765+0000 7efc17e00700 1 -- 172.21.15.46:0/3418140743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc18109c90 msgr2=0x7efc180834c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc17e00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc18109c90 0x7efc180834c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc17e00700 1 -- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1810e3c0 msgr2=0x7efc1807c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:35.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc17e00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1810e3c0 0x7efc1807c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc17e00700 1 -- 172.21.15.46:0/3418140743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1807ca80 con 0x7efc181092e0 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc17400700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc18109c90 0x7efc180834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc17e00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc181092e0 0x7efc18082f80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc08003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc14c00700 1 -- 172.21.15.46:0/3418140743 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08003680 con 0x7efc181092e0 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1807cd10 con 0x7efc181092e0 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc14c00700 1 -- 172.21.15.46:0/3418140743 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc08003e50 con 0x7efc181092e0 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc14c00700 1 -- 172.21.15.46:0/3418140743 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0800fe10 con 0x7efc181092e0 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.766+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc18080c80 con 0x7efc181092e0 2024-04-17T07:57:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.768+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1810a7c0 con 0x7efc181092e0 2024-04-17T07:57:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.770+0000 7efc14c00700 1 -- 172.21.15.46:0/3418140743 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc0800f460 con 0x7efc181092e0 2024-04-17T07:57:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.771+0000 7efc14c00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efbfc0a3de0 0x7efbfc0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.771+0000 7efc14c00700 1 -- 172.21.15.46:0/3418140743 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7efc080d1410 con 0x7efc181092e0 2024-04-17T07:57:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.771+0000 7efc17400700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efbfc0a3de0 0x7efbfc0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.772+0000 7efc17400700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efbfc0a3de0 0x7efbfc0a62a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc1807d7d0 tx=0x7efc04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:35.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:35.776+0000 7efc14c00700 1 -- 172.21.15.46:0/3418140743 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efc08013070 con 0x7efc181092e0 2024-04-17T07:57:36.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.002+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc18080ee0 con 0x7efbfc0a3de0 2024-04-17T07:57:36.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.020+0000 7efc14c00700 1 -- 172.21.15.46:0/3418140743 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340493 (secure 0 0 0) 0x7efc18080ee0 con 0x7efbfc0a3de0 2024-04-17T07:57:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efbfc0a3de0 msgr2=0x7efbfc0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efbfc0a3de0 0x7efbfc0a62a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc1807d7d0 tx=0x7efc04009260 comp rx=0 tx=0).stop 2024-04-17T07:57:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc181092e0 msgr2=0x7efc18082f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc181092e0 0x7efc18082f80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc08003a90 comp rx=0 tx=0).stop 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 shutdown_connections 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efbfc0a3de0 0x7efbfc0a62a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1810e3c0 0x7efc1807c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc181092e0 0x7efc18082f80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 --2- 172.21.15.46:0/3418140743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc18109c90 0x7efc180834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 >> 172.21.15.46:0/3418140743 conn(0x7efc18076220 msgr2=0x7efc18077850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 shutdown_connections 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:36.026+0000 7efc1ee00700 1 -- 172.21.15.46:0/3418140743 wait complete. 2024-04-17T07:57:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:57:36.967 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-04-17T07:57:35.560380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658150,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":740,"num_read_kb":490,"num_write":357,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181973,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380352,"kb_used_data":14480,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561472,"statfs":{"total":1151940427776,"available":1151550947328,"internally_reserved":0,"allocated":14827520,"data_stored":8185448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":31,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004641"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987034+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:57:22.987034+0000","last_peered":"2024-04-17T07:57:22.987034+0000","last_clean":"2024-04-17T07:57:22.987034+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:57:22.987034+0000","last_undegraded":"2024-04-17T07:57:22.987034+0000","last_fullsized":"2024-04-17T07:57:22.987034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301182+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:57:23.301182+0000","last_peered":"2024-04-17T07:57:23.301182+0000","last_clean":"2024-04-17T07:57:23.301182+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:57:23.301182+0000","last_undegraded":"2024-04-17T07:57:23.301182+0000","last_fullsized":"2024-04-17T07:57:23.301182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300823+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:57:23.300823+0000","last_peered":"2024-04-17T07:57:23.300823+0000","last_clean":"2024-04-17T07:57:23.300823+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:57:23.300823+0000","last_undegraded":"2024-04-17T07:57:23.300823+0000","last_fullsized":"2024-04-17T07:57:23.300823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987715+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:57:22.987715+0000","last_peered":"2024-04-17T07:57:22.987715+0000","last_clean":"2024-04-17T07:57:22.987715+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:57:22.987715+0000","last_undegraded":"2024-04-17T07:57:22.987715+0000","last_fullsized":"2024-04-17T07:57:22.987715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989568+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:57:22.989568+0000","last_peered":"2024-04-17T07:57:22.989568+0000","last_clean":"2024-04-17T07:57:22.989568+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:57:22.989568+0000","last_undegraded":"2024-04-17T07:57:22.989568+0000","last_fullsized":"2024-04-17T07:57:22.989568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990211+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:57:22.990211+0000","last_peered":"2024-04-17T07:57:22.990211+0000","last_clean":"2024-04-17T07:57:22.990211+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:57:22.990211+0000","last_undegraded":"2024-04-17T07:57:22.990211+0000","last_fullsized":"2024-04-17T07:57:22.990211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989436+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:57:22.989436+0000","last_peered":"2024-04-17T07:57:22.989436+0000","last_clean":"2024-04-17T07:57:22.989436+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:57:22.989436+0000","last_undegraded":"2024-04-17T07:57:22.989436+0000","last_fullsized":"2024-04-17T07:57:22.989436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981303+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:57:22.981303+0000","last_peered":"2024-04-17T07:57:22.981303+0000","last_clean":"2024-04-17T07:57:22.981303+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:57:22.981303+0000","last_undegraded":"2024-04-17T07:57:22.981303+0000","last_fullsized":"2024-04-17T07:57:22.981303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298370+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:57:23.298370+0000","last_peered":"2024-04-17T07:57:23.298370+0000","last_clean":"2024-04-17T07:57:23.298370+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:57:23.298370+0000","last_undegraded":"2024-04-17T07:57:23.298370+0000","last_fullsized":"2024-04-17T07:57:23.298370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991013+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:57:22.991013+0000","last_peered":"2024-04-17T07:57:22.991013+0000","last_clean":"2024-04-17T07:57:22.991013+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:57:22.991013+0000","last_undegraded":"2024-04-17T07:57:22.991013+0000","last_fullsized":"2024-04-17T07:57:22.991013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990203+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:57:22.990203+0000","last_peered":"2024-04-17T07:57:22.990203+0000","last_clean":"2024-04-17T07:57:22.990203+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:57:22.990203+0000","last_undegraded":"2024-04-17T07:57:22.990203+0000","last_fullsized":"2024-04-17T07:57:22.990203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987774+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:57:22.987774+0000","last_peered":"2024-04-17T07:57:22.987774+0000","last_clean":"2024-04-17T07:57:22.987774+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:57:22.987774+0000","last_undegraded":"2024-04-17T07:57:22.987774+0000","last_fullsized":"2024-04-17T07:57:22.987774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984322+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:57:22.984322+0000","last_peered":"2024-04-17T07:57:22.984322+0000","last_clean":"2024-04-17T07:57:22.984322+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:57:22.984322+0000","last_undegraded":"2024-04-17T07:57:22.984322+0000","last_fullsized":"2024-04-17T07:57:22.984322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300112+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:57:23.300112+0000","last_peered":"2024-04-17T07:57:23.300112+0000","last_clean":"2024-04-17T07:57:23.300112+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:57:23.300112+0000","last_undegraded":"2024-04-17T07:57:23.300112+0000","last_fullsized":"2024-04-17T07:57:23.300112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300251+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:57:23.300251+0000","last_peered":"2024-04-17T07:57:23.300251+0000","last_clean":"2024-04-17T07:57:23.300251+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:57:23.300251+0000","last_undegraded":"2024-04-17T07:57:23.300251+0000","last_fullsized":"2024-04-17T07:57:23.300251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297603+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:57:23.297603+0000","last_peered":"2024-04-17T07:57:23.297603+0000","last_clean":"2024-04-17T07:57:23.297603+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:57:23.297603+0000","last_undegraded":"2024-04-17T07:57:23.297603+0000","last_fullsized":"2024-04-17T07:57:23.297603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986826+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:57:22.986826+0000","last_peered":"2024-04-17T07:57:22.986826+0000","last_clean":"2024-04-17T07:57:22.986826+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:57:22.986826+0000","last_undegraded":"2024-04-17T07:57:22.986826+0000","last_fullsized":"2024-04-17T07:57:22.986826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299034+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:57:23.299034+0000","last_peered":"2024-04-17T07:57:23.299034+0000","last_clean":"2024-04-17T07:57:23.299034+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:57:23.299034+0000","last_undegraded":"2024-04-17T07:57:23.299034+0000","last_fullsized":"2024-04-17T07:57:23.299034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990468+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:57:22.990468+0000","last_peered":"2024-04-17T07:57:22.990468+0000","last_clean":"2024-04-17T07:57:22.990468+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:57:22.990468+0000","last_undegraded":"2024-04-17T07:57:22.990468+0000","last_fullsized":"2024-04-17T07:57:22.990468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990092+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:57:22.990092+0000","last_peered":"2024-04-17T07:57:22.990092+0000","last_clean":"2024-04-17T07:57:22.990092+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:57:22.990092+0000","last_undegraded":"2024-04-17T07:57:22.990092+0000","last_fullsized":"2024-04-17T07:57:22.990092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299637+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:57:23.299637+0000","last_peered":"2024-04-17T07:57:23.299637+0000","last_clean":"2024-04-17T07:57:23.299637+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:57:23.299637+0000","last_undegraded":"2024-04-17T07:57:23.299637+0000","last_fullsized":"2024-04-17T07:57:23.299637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298902+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:57:23.298902+0000","last_peered":"2024-04-17T07:57:23.298902+0000","last_clean":"2024-04-17T07:57:23.298902+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:57:23.298902+0000","last_undegraded":"2024-04-17T07:57:23.298902+0000","last_fullsized":"2024-04-17T07:57:23.298902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987051+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:57:22.987051+0000","last_peered":"2024-04-17T07:57:22.987051+0000","last_clean":"2024-04-17T07:57:22.987051+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:57:22.987051+0000","last_undegraded":"2024-04-17T07:57:22.987051+0000","last_fullsized":"2024-04-17T07:57:22.987051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984617+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:57:22.984617+0000","last_peered":"2024-04-17T07:57:22.984617+0000","last_clean":"2024-04-17T07:57:22.984617+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:57:22.984617+0000","last_undegraded":"2024-04-17T07:57:22.984617+0000","last_fullsized":"2024-04-17T07:57:22.984617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301222+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:57:23.301222+0000","last_peered":"2024-04-17T07:57:23.301222+0000","last_clean":"2024-04-17T07:57:23.301222+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:57:23.301222+0000","last_undegraded":"2024-04-17T07:57:23.301222+0000","last_fullsized":"2024-04-17T07:57:23.301222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987157+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:57:22.987157+0000","last_peered":"2024-04-17T07:57:22.987157+0000","last_clean":"2024-04-17T07:57:22.987157+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:57:22.987157+0000","last_undegraded":"2024-04-17T07:57:22.987157+0000","last_fullsized":"2024-04-17T07:57:22.987157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.996476+0000","last_change":"2024-04-17T07:57:22.996476+0000","last_active":"2024-04-17T07:57:22.996476+0000","last_peered":"2024-04-17T07:57:22.996476+0000","last_clean":"2024-04-17T07:57:22.996476+0000","last_became_active":"2024-04-17T07:57:22.991654+0000","last_became_peered":"2024-04-17T07:57:22.991654+0000","last_unstale":"2024-04-17T07:57:22.996476+0000","last_undegraded":"2024-04-17T07:57:22.996476+0000","last_fullsized":"2024-04-17T07:57:22.996476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:33.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298000+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:57:23.298000+0000","last_peered":"2024-04-17T07:57:23.298000+0000","last_clean":"2024-04-17T07:57:23.298000+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:57:23.298000+0000","last_undegraded":"2024-04-17T07:57:23.298000+0000","last_fullsized":"2024-04-17T07:57:23.298000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987060+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:57:22.987060+0000","last_peered":"2024-04-17T07:57:22.987060+0000","last_clean":"2024-04-17T07:57:22.987060+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:57:22.987060+0000","last_undegraded":"2024-04-17T07:57:22.987060+0000","last_fullsized":"2024-04-17T07:57:22.987060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990006+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:57:22.990006+0000","last_peered":"2024-04-17T07:57:22.990006+0000","last_clean":"2024-04-17T07:57:22.990006+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:57:22.990006+0000","last_undegraded":"2024-04-17T07:57:22.990006+0000","last_fullsized":"2024-04-17T07:57:22.990006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981957+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:57:22.981957+0000","last_peered":"2024-04-17T07:57:22.981957+0000","last_clean":"2024-04-17T07:57:22.981957+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:57:22.981957+0000","last_undegraded":"2024-04-17T07:57:22.981957+0000","last_fullsized":"2024-04-17T07:57:22.981957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990345+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:57:22.990345+0000","last_peered":"2024-04-17T07:57:22.990345+0000","last_clean":"2024-04-17T07:57:22.990345+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:57:22.990345+0000","last_undegraded":"2024-04-17T07:57:22.990345+0000","last_fullsized":"2024-04-17T07:57:22.990345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":139,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:29.688566+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:57:29.688566+0000","last_peered":"2024-04-17T07:57:29.688566+0000","last_clean":"2024-04-17T07:57:29.688566+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:57:29.688566+0000","last_undegraded":"2024-04-17T07:57:29.688566+0000","last_fullsized":"2024-04-17T07:57:29.688566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989890+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:57:22.989890+0000","last_peered":"2024-04-17T07:57:22.989890+0000","last_clean":"2024-04-17T07:57:22.989890+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:57:22.989890+0000","last_undegraded":"2024-04-17T07:57:22.989890+0000","last_fullsized":"2024-04-17T07:57:22.989890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989695+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:57:22.989695+0000","last_peered":"2024-04-17T07:57:22.989695+0000","last_clean":"2024-04-17T07:57:22.989695+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:57:22.989695+0000","last_undegraded":"2024-04-17T07:57:22.989695+0000","last_fullsized":"2024-04-17T07:57:22.989695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990075+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:57:22.990075+0000","last_peered":"2024-04-17T07:57:22.990075+0000","last_clean":"2024-04-17T07:57:22.990075+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:57:22.990075+0000","last_undegraded":"2024-04-17T07:57:22.990075+0000","last_fullsized":"2024-04-17T07:57:22.990075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981716+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:57:22.981716+0000","last_peered":"2024-04-17T07:57:22.981716+0000","last_clean":"2024-04-17T07:57:22.981716+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:57:22.981716+0000","last_undegraded":"2024-04-17T07:57:22.981716+0000","last_fullsized":"2024-04-17T07:57:22.981716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986829+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:57:22.986829+0000","last_peered":"2024-04-17T07:57:22.986829+0000","last_clean":"2024-04-17T07:57:22.986829+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:57:22.986829+0000","last_undegraded":"2024-04-17T07:57:22.986829+0000","last_fullsized":"2024-04-17T07:57:22.986829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982797+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:57:22.982797+0000","last_peered":"2024-04-17T07:57:22.982797+0000","last_clean":"2024-04-17T07:57:22.982797+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:57:22.982797+0000","last_undegraded":"2024-04-17T07:57:22.982797+0000","last_fullsized":"2024-04-17T07:57:22.982797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.980936+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:57:22.980936+0000","last_peered":"2024-04-17T07:57:22.980936+0000","last_clean":"2024-04-17T07:57:22.980936+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:57:22.980936+0000","last_undegraded":"2024-04-17T07:57:22.980936+0000","last_fullsized":"2024-04-17T07:57:22.980936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301002+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:57:23.301002+0000","last_peered":"2024-04-17T07:57:23.301002+0000","last_clean":"2024-04-17T07:57:23.301002+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:57:23.301002+0000","last_undegraded":"2024-04-17T07:57:23.301002+0000","last_fullsized":"2024-04-17T07:57:23.301002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981394+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:57:22.981394+0000","last_peered":"2024-04-17T07:57:22.981394+0000","last_clean":"2024-04-17T07:57:22.981394+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:57:22.981394+0000","last_undegraded":"2024-04-17T07:57:22.981394+0000","last_fullsized":"2024-04-17T07:57:22.981394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299000+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:57:23.299000+0000","last_peered":"2024-04-17T07:57:23.299000+0000","last_clean":"2024-04-17T07:57:23.299000+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:57:23.299000+0000","last_undegraded":"2024-04-17T07:57:23.299000+0000","last_fullsized":"2024-04-17T07:57:23.299000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981197+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:57:22.981197+0000","last_peered":"2024-04-17T07:57:22.981197+0000","last_clean":"2024-04-17T07:57:22.981197+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:57:22.981197+0000","last_undegraded":"2024-04-17T07:57:22.981197+0000","last_fullsized":"2024-04-17T07:57:22.981197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":133,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:30.689780+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:57:30.689780+0000","last_peered":"2024-04-17T07:57:30.689780+0000","last_clean":"2024-04-17T07:57:30.689780+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:57:30.689780+0000","last_undegraded":"2024-04-17T07:57:30.689780+0000","last_fullsized":"2024-04-17T07:57:30.689780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298379+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:57:23.298379+0000","last_peered":"2024-04-17T07:57:23.298379+0000","last_clean":"2024-04-17T07:57:23.298379+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:57:23.298379+0000","last_undegraded":"2024-04-17T07:57:23.298379+0000","last_fullsized":"2024-04-17T07:57:23.298379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299142+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:57:23.299142+0000","last_peered":"2024-04-17T07:57:23.299142+0000","last_clean":"2024-04-17T07:57:23.299142+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:57:23.299142+0000","last_undegraded":"2024-04-17T07:57:23.299142+0000","last_fullsized":"2024-04-17T07:57:23.299142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989795+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:57:22.989795+0000","last_peered":"2024-04-17T07:57:22.989795+0000","last_clean":"2024-04-17T07:57:22.989795+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:57:22.989795+0000","last_undegraded":"2024-04-17T07:57:22.989795+0000","last_fullsized":"2024-04-17T07:57:22.989795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301097+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:57:23.301097+0000","last_peered":"2024-04-17T07:57:23.301097+0000","last_clean":"2024-04-17T07:57:23.301097+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:57:23.301097+0000","last_undegraded":"2024-04-17T07:57:23.301097+0000","last_fullsized":"2024-04-17T07:57:23.301097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987275+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:57:22.987275+0000","last_peered":"2024-04-17T07:57:22.987275+0000","last_clean":"2024-04-17T07:57:22.987275+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:57:22.987275+0000","last_undegraded":"2024-04-17T07:57:22.987275+0000","last_fullsized":"2024-04-17T07:57:22.987275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981928+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:57:22.981928+0000","last_peered":"2024-04-17T07:57:22.981928+0000","last_clean":"2024-04-17T07:57:22.981928+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:57:22.981928+0000","last_undegraded":"2024-04-17T07:57:22.981928+0000","last_fullsized":"2024-04-17T07:57:22.981928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298113+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:57:23.298113+0000","last_peered":"2024-04-17T07:57:23.298113+0000","last_clean":"2024-04-17T07:57:23.298113+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:57:23.298113+0000","last_undegraded":"2024-04-17T07:57:23.298113+0000","last_fullsized":"2024-04-17T07:57:23.298113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298615+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:57:23.298615+0000","last_peered":"2024-04-17T07:57:23.298615+0000","last_clean":"2024-04-17T07:57:23.298615+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:57:23.298615+0000","last_undegraded":"2024-04-17T07:57:23.298615+0000","last_fullsized":"2024-04-17T07:57:23.298615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":271,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:31.691686+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:57:31.691686+0000","last_peered":"2024-04-17T07:57:31.691686+0000","last_clean":"2024-04-17T07:57:31.691686+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:57:31.691686+0000","last_undegraded":"2024-04-17T07:57:31.691686+0000","last_fullsized":"2024-04-17T07:57:31.691686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":165,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991075+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:57:22.991075+0000","last_peered":"2024-04-17T07:57:22.991075+0000","last_clean":"2024-04-17T07:57:22.991075+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:57:22.991075+0000","last_undegraded":"2024-04-17T07:57:22.991075+0000","last_fullsized":"2024-04-17T07:57:22.991075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982168+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:57:22.982168+0000","last_peered":"2024-04-17T07:57:22.982168+0000","last_clean":"2024-04-17T07:57:22.982168+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:57:22.982168+0000","last_undegraded":"2024-04-17T07:57:22.982168+0000","last_fullsized":"2024-04-17T07:57:22.982168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989392+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:57:22.989392+0000","last_peered":"2024-04-17T07:57:22.989392+0000","last_clean":"2024-04-17T07:57:22.989392+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:57:22.989392+0000","last_undegraded":"2024-04-17T07:57:22.989392+0000","last_fullsized":"2024-04-17T07:57:22.989392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983358+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:57:22.983358+0000","last_peered":"2024-04-17T07:57:22.983358+0000","last_clean":"2024-04-17T07:57:22.983358+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:57:22.983358+0000","last_undegraded":"2024-04-17T07:57:22.983358+0000","last_fullsized":"2024-04-17T07:57:22.983358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986309+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:57:22.986309+0000","last_peered":"2024-04-17T07:57:22.986309+0000","last_clean":"2024-04-17T07:57:22.986309+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:57:22.986309+0000","last_undegraded":"2024-04-17T07:57:22.986309+0000","last_fullsized":"2024-04-17T07:57:22.986309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981818+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:57:22.981818+0000","last_peered":"2024-04-17T07:57:22.981818+0000","last_clean":"2024-04-17T07:57:22.981818+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:57:22.981818+0000","last_undegraded":"2024-04-17T07:57:22.981818+0000","last_fullsized":"2024-04-17T07:57:22.981818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989572+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:57:22.989572+0000","last_peered":"2024-04-17T07:57:22.989572+0000","last_clean":"2024-04-17T07:57:22.989572+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:57:22.989572+0000","last_undegraded":"2024-04-17T07:57:22.989572+0000","last_fullsized":"2024-04-17T07:57:22.989572+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299360+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:57:23.299360+0000","last_peered":"2024-04-17T07:57:23.299360+0000","last_clean":"2024-04-17T07:57:23.299360+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:57:23.299360+0000","last_undegraded":"2024-04-17T07:57:23.299360+0000","last_fullsized":"2024-04-17T07:57:23.299360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:30.010249+0000","last_change":"2024-04-17T07:57:22.995880+0000","last_active":"2024-04-17T07:57:30.010249+0000","last_peered":"2024-04-17T07:57:30.010249+0000","last_clean":"2024-04-17T07:57:30.010249+0000","last_became_active":"2024-04-17T07:57:22.991967+0000","last_became_peered":"2024-04-17T07:57:22.991967+0000","last_unstale":"2024-04-17T07:57:30.010249+0000","last_undegraded":"2024-04-17T07:57:30.010249+0000","last_fullsized":"2024-04-17T07:57:30.010249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:27.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989483+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:57:22.989483+0000","last_peered":"2024-04-17T07:57:22.989483+0000","last_clean":"2024-04-17T07:57:22.989483+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:57:22.989483+0000","last_undegraded":"2024-04-17T07:57:22.989483+0000","last_fullsized":"2024-04-17T07:57:22.989483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984496+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:57:22.984496+0000","last_peered":"2024-04-17T07:57:22.984496+0000","last_clean":"2024-04-17T07:57:22.984496+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:57:22.984496+0000","last_undegraded":"2024-04-17T07:57:22.984496+0000","last_fullsized":"2024-04-17T07:57:22.984496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989772+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:57:22.989772+0000","last_peered":"2024-04-17T07:57:22.989772+0000","last_clean":"2024-04-17T07:57:22.989772+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:57:22.989772+0000","last_undegraded":"2024-04-17T07:57:22.989772+0000","last_fullsized":"2024-04-17T07:57:22.989772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981395+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:57:22.981395+0000","last_peered":"2024-04-17T07:57:22.981395+0000","last_clean":"2024-04-17T07:57:22.981395+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:57:22.981395+0000","last_undegraded":"2024-04-17T07:57:22.981395+0000","last_fullsized":"2024-04-17T07:57:22.981395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981314+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:57:22.981314+0000","last_peered":"2024-04-17T07:57:22.981314+0000","last_clean":"2024-04-17T07:57:22.981314+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:57:22.981314+0000","last_undegraded":"2024-04-17T07:57:22.981314+0000","last_fullsized":"2024-04-17T07:57:22.981314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991206+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:57:22.991206+0000","last_peered":"2024-04-17T07:57:22.991206+0000","last_clean":"2024-04-17T07:57:22.991206+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:57:22.991206+0000","last_undegraded":"2024-04-17T07:57:22.991206+0000","last_fullsized":"2024-04-17T07:57:22.991206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986817+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:57:22.986817+0000","last_peered":"2024-04-17T07:57:22.986817+0000","last_clean":"2024-04-17T07:57:22.986817+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:57:22.986817+0000","last_undegraded":"2024-04-17T07:57:22.986817+0000","last_fullsized":"2024-04-17T07:57:22.986817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989642+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:57:22.989642+0000","last_peered":"2024-04-17T07:57:22.989642+0000","last_clean":"2024-04-17T07:57:22.989642+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:57:22.989642+0000","last_undegraded":"2024-04-17T07:57:22.989642+0000","last_fullsized":"2024-04-17T07:57:22.989642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299647+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:57:23.299647+0000","last_peered":"2024-04-17T07:57:23.299647+0000","last_clean":"2024-04-17T07:57:23.299647+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:57:23.299647+0000","last_undegraded":"2024-04-17T07:57:23.299647+0000","last_fullsized":"2024-04-17T07:57:23.299647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982460+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:57:22.982460+0000","last_peered":"2024-04-17T07:57:22.982460+0000","last_clean":"2024-04-17T07:57:22.982460+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:57:22.982460+0000","last_undegraded":"2024-04-17T07:57:22.982460+0000","last_fullsized":"2024-04-17T07:57:22.982460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989416+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:57:22.989416+0000","last_peered":"2024-04-17T07:57:22.989416+0000","last_clean":"2024-04-17T07:57:22.989416+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:57:22.989416+0000","last_undegraded":"2024-04-17T07:57:22.989416+0000","last_fullsized":"2024-04-17T07:57:22.989416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990564+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:57:22.990564+0000","last_peered":"2024-04-17T07:57:22.990564+0000","last_clean":"2024-04-17T07:57:22.990564+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:57:22.990564+0000","last_undegraded":"2024-04-17T07:57:22.990564+0000","last_fullsized":"2024-04-17T07:57:22.990564+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.985641+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:57:22.985641+0000","last_peered":"2024-04-17T07:57:22.985641+0000","last_clean":"2024-04-17T07:57:22.985641+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:57:22.985641+0000","last_undegraded":"2024-04-17T07:57:22.985641+0000","last_fullsized":"2024-04-17T07:57:22.985641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":134,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:31.011716+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:57:31.011716+0000","last_peered":"2024-04-17T07:57:31.011716+0000","last_clean":"2024-04-17T07:57:31.011716+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:57:31.011716+0000","last_undegraded":"2024-04-17T07:57:31.011716+0000","last_fullsized":"2024-04-17T07:57:31.011716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989145+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:57:22.989145+0000","last_peered":"2024-04-17T07:57:22.989145+0000","last_clean":"2024-04-17T07:57:22.989145+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:57:22.989145+0000","last_undegraded":"2024-04-17T07:57:22.989145+0000","last_fullsized":"2024-04-17T07:57:22.989145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982675+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:57:22.982675+0000","last_peered":"2024-04-17T07:57:22.982675+0000","last_clean":"2024-04-17T07:57:22.982675+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:57:22.982675+0000","last_undegraded":"2024-04-17T07:57:22.982675+0000","last_fullsized":"2024-04-17T07:57:22.982675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987305+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:57:22.987305+0000","last_peered":"2024-04-17T07:57:22.987305+0000","last_clean":"2024-04-17T07:57:22.987305+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:57:22.987305+0000","last_undegraded":"2024-04-17T07:57:22.987305+0000","last_fullsized":"2024-04-17T07:57:22.987305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989188+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:57:22.989188+0000","last_peered":"2024-04-17T07:57:22.989188+0000","last_clean":"2024-04-17T07:57:22.989188+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:57:22.989188+0000","last_undegraded":"2024-04-17T07:57:22.989188+0000","last_fullsized":"2024-04-17T07:57:22.989188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982007+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:57:22.982007+0000","last_peered":"2024-04-17T07:57:22.982007+0000","last_clean":"2024-04-17T07:57:22.982007+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:57:22.982007+0000","last_undegraded":"2024-04-17T07:57:22.982007+0000","last_fullsized":"2024-04-17T07:57:22.982007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982417+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:57:22.982417+0000","last_peered":"2024-04-17T07:57:22.982417+0000","last_clean":"2024-04-17T07:57:22.982417+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:57:22.982417+0000","last_undegraded":"2024-04-17T07:57:22.982417+0000","last_fullsized":"2024-04-17T07:57:22.982417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989408+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:57:22.989408+0000","last_peered":"2024-04-17T07:57:22.989408+0000","last_clean":"2024-04-17T07:57:22.989408+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:57:22.989408+0000","last_undegraded":"2024-04-17T07:57:22.989408+0000","last_fullsized":"2024-04-17T07:57:22.989408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986287+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:57:22.986287+0000","last_peered":"2024-04-17T07:57:22.986287+0000","last_clean":"2024-04-17T07:57:22.986287+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:57:22.986287+0000","last_undegraded":"2024-04-17T07:57:22.986287+0000","last_fullsized":"2024-04-17T07:57:22.986287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983466+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:57:22.983466+0000","last_peered":"2024-04-17T07:57:22.983466+0000","last_clean":"2024-04-17T07:57:22.983466+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:57:22.983466+0000","last_undegraded":"2024-04-17T07:57:22.983466+0000","last_fullsized":"2024-04-17T07:57:22.983466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981116+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:57:22.981116+0000","last_peered":"2024-04-17T07:57:22.981116+0000","last_clean":"2024-04-17T07:57:22.981116+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:57:22.981116+0000","last_undegraded":"2024-04-17T07:57:22.981116+0000","last_fullsized":"2024-04-17T07:57:22.981116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299253+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:57:23.299253+0000","last_peered":"2024-04-17T07:57:23.299253+0000","last_clean":"2024-04-17T07:57:23.299253+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:57:23.299253+0000","last_undegraded":"2024-04-17T07:57:23.299253+0000","last_fullsized":"2024-04-17T07:57:23.299253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984112+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:57:22.984112+0000","last_peered":"2024-04-17T07:57:22.984112+0000","last_clean":"2024-04-17T07:57:22.984112+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:57:22.984112+0000","last_undegraded":"2024-04-17T07:57:22.984112+0000","last_fullsized":"2024-04-17T07:57:22.984112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989763+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:57:22.989763+0000","last_peered":"2024-04-17T07:57:22.989763+0000","last_clean":"2024-04-17T07:57:22.989763+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:57:22.989763+0000","last_undegraded":"2024-04-17T07:57:22.989763+0000","last_fullsized":"2024-04-17T07:57:22.989763+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987850+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:57:22.987850+0000","last_peered":"2024-04-17T07:57:22.987850+0000","last_clean":"2024-04-17T07:57:22.987850+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:57:22.987850+0000","last_undegraded":"2024-04-17T07:57:22.987850+0000","last_fullsized":"2024-04-17T07:57:22.987850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301271+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:57:23.301271+0000","last_peered":"2024-04-17T07:57:23.301271+0000","last_clean":"2024-04-17T07:57:23.301271+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:57:23.301271+0000","last_undegraded":"2024-04-17T07:57:23.301271+0000","last_fullsized":"2024-04-17T07:57:23.301271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987589+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:57:22.987589+0000","last_peered":"2024-04-17T07:57:22.987589+0000","last_clean":"2024-04-17T07:57:22.987589+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:57:22.987589+0000","last_undegraded":"2024-04-17T07:57:22.987589+0000","last_fullsized":"2024-04-17T07:57:22.987589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297678+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:57:23.297678+0000","last_peered":"2024-04-17T07:57:23.297678+0000","last_clean":"2024-04-17T07:57:23.297678+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:57:23.297678+0000","last_undegraded":"2024-04-17T07:57:23.297678+0000","last_fullsized":"2024-04-17T07:57:23.297678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989036+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:57:22.989036+0000","last_peered":"2024-04-17T07:57:22.989036+0000","last_clean":"2024-04-17T07:57:22.989036+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:57:22.989036+0000","last_undegraded":"2024-04-17T07:57:22.989036+0000","last_fullsized":"2024-04-17T07:57:22.989036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299526+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:57:23.299526+0000","last_peered":"2024-04-17T07:57:23.299526+0000","last_clean":"2024-04-17T07:57:23.299526+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:57:23.299526+0000","last_undegraded":"2024-04-17T07:57:23.299526+0000","last_fullsized":"2024-04-17T07:57:23.299526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981526+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:57:22.981526+0000","last_peered":"2024-04-17T07:57:22.981526+0000","last_clean":"2024-04-17T07:57:22.981526+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:57:22.981526+0000","last_undegraded":"2024-04-17T07:57:22.981526+0000","last_fullsized":"2024-04-17T07:57:22.981526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986228+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:57:22.986228+0000","last_peered":"2024-04-17T07:57:22.986228+0000","last_clean":"2024-04-17T07:57:22.986228+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:57:22.986228+0000","last_undegraded":"2024-04-17T07:57:22.986228+0000","last_fullsized":"2024-04-17T07:57:22.986228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987327+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:57:22.987327+0000","last_peered":"2024-04-17T07:57:22.987327+0000","last_clean":"2024-04-17T07:57:22.987327+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:57:22.987327+0000","last_undegraded":"2024-04-17T07:57:22.987327+0000","last_fullsized":"2024-04-17T07:57:22.987327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990287+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:57:22.990287+0000","last_peered":"2024-04-17T07:57:22.990287+0000","last_clean":"2024-04-17T07:57:22.990287+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:57:22.990287+0000","last_undegraded":"2024-04-17T07:57:22.990287+0000","last_fullsized":"2024-04-17T07:57:22.990287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983499+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:57:22.983499+0000","last_peered":"2024-04-17T07:57:22.983499+0000","last_clean":"2024-04-17T07:57:22.983499+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:57:22.983499+0000","last_undegraded":"2024-04-17T07:57:22.983499+0000","last_fullsized":"2024-04-17T07:57:22.983499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987653+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:57:22.987653+0000","last_peered":"2024-04-17T07:57:22.987653+0000","last_clean":"2024-04-17T07:57:22.987653+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:57:22.987653+0000","last_undegraded":"2024-04-17T07:57:22.987653+0000","last_fullsized":"2024-04-17T07:57:22.987653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989692+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:57:22.989692+0000","last_peered":"2024-04-17T07:57:22.989692+0000","last_clean":"2024-04-17T07:57:22.989692+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:57:22.989692+0000","last_undegraded":"2024-04-17T07:57:22.989692+0000","last_fullsized":"2024-04-17T07:57:22.989692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989471+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:57:22.989471+0000","last_peered":"2024-04-17T07:57:22.989471+0000","last_clean":"2024-04-17T07:57:22.989471+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:57:22.989471+0000","last_undegraded":"2024-04-17T07:57:22.989471+0000","last_fullsized":"2024-04-17T07:57:22.989471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297723+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:57:23.297723+0000","last_peered":"2024-04-17T07:57:23.297723+0000","last_clean":"2024-04-17T07:57:23.297723+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:57:23.297723+0000","last_undegraded":"2024-04-17T07:57:23.297723+0000","last_fullsized":"2024-04-17T07:57:23.297723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989968+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:57:22.989968+0000","last_peered":"2024-04-17T07:57:22.989968+0000","last_clean":"2024-04-17T07:57:22.989968+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:57:22.989968+0000","last_undegraded":"2024-04-17T07:57:22.989968+0000","last_fullsized":"2024-04-17T07:57:22.989968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990328+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:57:22.990328+0000","last_peered":"2024-04-17T07:57:22.990328+0000","last_clean":"2024-04-17T07:57:22.990328+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:57:22.990328+0000","last_undegraded":"2024-04-17T07:57:22.990328+0000","last_fullsized":"2024-04-17T07:57:22.990328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299788+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:57:23.299788+0000","last_peered":"2024-04-17T07:57:23.299788+0000","last_clean":"2024-04-17T07:57:23.299788+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:57:23.299788+0000","last_undegraded":"2024-04-17T07:57:23.299788+0000","last_fullsized":"2024-04-17T07:57:23.299788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982080+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:57:22.982080+0000","last_peered":"2024-04-17T07:57:22.982080+0000","last_clean":"2024-04-17T07:57:22.982080+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:57:22.982080+0000","last_undegraded":"2024-04-17T07:57:22.982080+0000","last_fullsized":"2024-04-17T07:57:22.982080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989766+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:57:22.989766+0000","last_peered":"2024-04-17T07:57:22.989766+0000","last_clean":"2024-04-17T07:57:22.989766+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:57:22.989766+0000","last_undegraded":"2024-04-17T07:57:22.989766+0000","last_fullsized":"2024-04-17T07:57:22.989766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981178+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:57:22.981178+0000","last_peered":"2024-04-17T07:57:22.981178+0000","last_clean":"2024-04-17T07:57:22.981178+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:57:22.981178+0000","last_undegraded":"2024-04-17T07:57:22.981178+0000","last_fullsized":"2024-04-17T07:57:22.981178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984793+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:57:22.984793+0000","last_peered":"2024-04-17T07:57:22.984793+0000","last_clean":"2024-04-17T07:57:22.984793+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:57:22.984793+0000","last_undegraded":"2024-04-17T07:57:22.984793+0000","last_fullsized":"2024-04-17T07:57:22.984793+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989884+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:57:22.989884+0000","last_peered":"2024-04-17T07:57:22.989884+0000","last_clean":"2024-04-17T07:57:22.989884+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:57:22.989884+0000","last_undegraded":"2024-04-17T07:57:22.989884+0000","last_fullsized":"2024-04-17T07:57:22.989884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986499+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:57:22.986499+0000","last_peered":"2024-04-17T07:57:22.986499+0000","last_clean":"2024-04-17T07:57:22.986499+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:57:22.986499+0000","last_undegraded":"2024-04-17T07:57:22.986499+0000","last_fullsized":"2024-04-17T07:57:22.986499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990103+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:57:22.990103+0000","last_peered":"2024-04-17T07:57:22.990103+0000","last_clean":"2024-04-17T07:57:22.990103+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:57:22.990103+0000","last_undegraded":"2024-04-17T07:57:22.990103+0000","last_fullsized":"2024-04-17T07:57:22.990103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990216+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:57:22.990216+0000","last_peered":"2024-04-17T07:57:22.990216+0000","last_clean":"2024-04-17T07:57:22.990216+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:57:22.990216+0000","last_undegraded":"2024-04-17T07:57:22.990216+0000","last_fullsized":"2024-04-17T07:57:22.990216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300391+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:57:23.300391+0000","last_peered":"2024-04-17T07:57:23.300391+0000","last_clean":"2024-04-17T07:57:23.300391+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:57:23.300391+0000","last_undegraded":"2024-04-17T07:57:23.300391+0000","last_fullsized":"2024-04-17T07:57:23.300391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983015+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:57:22.983015+0000","last_peered":"2024-04-17T07:57:22.983015+0000","last_clean":"2024-04-17T07:57:22.983015+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:57:22.983015+0000","last_undegraded":"2024-04-17T07:57:22.983015+0000","last_fullsized":"2024-04-17T07:57:22.983015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986662+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:57:22.986662+0000","last_peered":"2024-04-17T07:57:22.986662+0000","last_clean":"2024-04-17T07:57:22.986662+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:57:22.986662+0000","last_undegraded":"2024-04-17T07:57:22.986662+0000","last_fullsized":"2024-04-17T07:57:22.986662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990470+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:57:22.990470+0000","last_peered":"2024-04-17T07:57:22.990470+0000","last_clean":"2024-04-17T07:57:22.990470+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:57:22.990470+0000","last_undegraded":"2024-04-17T07:57:22.990470+0000","last_fullsized":"2024-04-17T07:57:22.990470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984574+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:57:22.984574+0000","last_peered":"2024-04-17T07:57:22.984574+0000","last_clean":"2024-04-17T07:57:22.984574+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:57:22.984574+0000","last_undegraded":"2024-04-17T07:57:22.984574+0000","last_fullsized":"2024-04-17T07:57:22.984574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981579+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:57:22.981579+0000","last_peered":"2024-04-17T07:57:22.981579+0000","last_clean":"2024-04-17T07:57:22.981579+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:57:22.981579+0000","last_undegraded":"2024-04-17T07:57:22.981579+0000","last_fullsized":"2024-04-17T07:57:22.981579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981814+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:57:22.981814+0000","last_peered":"2024-04-17T07:57:22.981814+0000","last_clean":"2024-04-17T07:57:22.981814+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:57:22.981814+0000","last_undegraded":"2024-04-17T07:57:22.981814+0000","last_fullsized":"2024-04-17T07:57:22.981814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297986+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:57:23.297986+0000","last_peered":"2024-04-17T07:57:23.297986+0000","last_clean":"2024-04-17T07:57:23.297986+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:57:23.297986+0000","last_undegraded":"2024-04-17T07:57:23.297986+0000","last_fullsized":"2024-04-17T07:57:23.297986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984520+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:57:22.984520+0000","last_peered":"2024-04-17T07:57:22.984520+0000","last_clean":"2024-04-17T07:57:22.984520+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:57:22.984520+0000","last_undegraded":"2024-04-17T07:57:22.984520+0000","last_fullsized":"2024-04-17T07:57:22.984520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986489+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:57:22.986489+0000","last_peered":"2024-04-17T07:57:22.986489+0000","last_clean":"2024-04-17T07:57:22.986489+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:57:22.986489+0000","last_undegraded":"2024-04-17T07:57:22.986489+0000","last_fullsized":"2024-04-17T07:57:22.986489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67548,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":614,"num_read_kb":381,"num_write":174,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1748992,"data_stored":1170263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":523921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":523841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1115207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":577785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":579667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1107287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:57:37.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:37 smithi062 ceph-mon[36507]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:37.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:37 smithi062 ceph-mon[36507]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:37.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:37 smithi046 ceph-mon[32038]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:37.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:37 smithi046 ceph-mon[32038]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:37 smithi053 ceph-mon[36437]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:37 smithi053 ceph-mon[36437]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:39.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:39 smithi046 ceph-mon[32038]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:39.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:39 smithi053 ceph-mon[36437]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:39.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:39 smithi062 ceph-mon[36507]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2024-04-17T07:57:39.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T07:57:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:41.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:41 smithi046 ceph-mon[32038]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:41.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:41 smithi053 ceph-mon[36437]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:41.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:41 smithi062 ceph-mon[36507]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 -- 172.21.15.46:0/2190436413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 msgr2=0x7f82bc102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 --2- 172.21.15.46:0/2190436413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc102e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009990 tx=0x7f82a8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 -- 172.21.15.46:0/2190436413 shutdown_connections 2024-04-17T07:57:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 --2- 172.21.15.46:0/2190436413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82bc103340 0x7f82bc10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 --2- 172.21.15.46:0/2190436413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108980 0x7f82bc108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 --2- 172.21.15.46:0/2190436413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc102e00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 -- 172.21.15.46:0/2190436413 >> 172.21.15.46:0/2190436413 conn(0x7f82bc0fe4d0 msgr2=0x7f82bc1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 -- 172.21.15.46:0/2190436413 shutdown_connections 2024-04-17T07:57:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.691+0000 7f82bbe00700 1 -- 172.21.15.46:0/2190436413 wait complete. 2024-04-17T07:57:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.692+0000 7f82bbe00700 1 Processor -- start 2024-04-17T07:57:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.692+0000 7f82bbe00700 1 -- start start 2024-04-17T07:57:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82bbe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82bbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc103340 0x7f82bc199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82bbe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82bc108980 0x7f82bc19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82bbe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc112d60 con 0x7f82bc103340 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82bbe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc112be0 con 0x7f82bc102980 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82bbe00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc112ee0 con 0x7f82bc108980 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82baa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82baa00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56916/0 (socket says 172.21.15.46:56916) 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82ba000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc103340 0x7f82bc199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.693+0000 7f82baa00700 1 -- 172.21.15.46:0/4043131113 learned_addr learned my addr 172.21.15.46:0/4043131113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82bb400700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82bc108980 0x7f82bc19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82baa00700 1 -- 172.21.15.46:0/4043131113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82bc108980 msgr2=0x7f82bc19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82baa00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82bc108980 0x7f82bc19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82baa00700 1 -- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc103340 msgr2=0x7f82bc199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82baa00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc103340 0x7f82bc199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82baa00700 1 -- 172.21.15.46:0/4043131113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc19e140 con 0x7f82bc102980 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82baa00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc199130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e9a0 tx=0x7f82ac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82b3400700 1 -- 172.21.15.46:0/4043131113 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00cbd0 con 0x7f82bc102980 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc19e430 con 0x7f82bc102980 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82b3400700 1 -- 172.21.15.46:0/4043131113 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ac00cd70 con 0x7f82bc102980 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.694+0000 7f82b3400700 1 -- 172.21.15.46:0/4043131113 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac018750 con 0x7f82bc102980 2024-04-17T07:57:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.695+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc1a5cb0 con 0x7f82bc102980 2024-04-17T07:57:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.696+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc069740 con 0x7f82bc102980 2024-04-17T07:57:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.698+0000 7f82b3400700 1 -- 172.21.15.46:0/4043131113 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f82ac010430 con 0x7f82bc102980 2024-04-17T07:57:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.700+0000 7f82b3400700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82a00a3c40 0x7f82a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.700+0000 7f82b3400700 1 -- 172.21.15.46:0/4043131113 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f82ac0d0a10 con 0x7f82bc102980 2024-04-17T07:57:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.700+0000 7f82ba000700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82a00a3c40 0x7f82a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.700+0000 7f82ba000700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82a00a3c40 0x7f82a00a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82bc100a90 tx=0x7f82a800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.702+0000 7f82b3400700 1 -- 172.21.15.46:0/4043131113 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f82ac09d000 con 0x7f82bc102980 2024-04-17T07:57:42.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.931+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82bc062e90 con 0x7f82a00a3c40 2024-04-17T07:57:42.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.942+0000 7f82b3400700 1 -- 172.21.15.46:0/4043131113 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340490 (secure 0 0 0) 0x7f82bc062e90 con 0x7f82a00a3c40 2024-04-17T07:57:42.947 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82a00a3c40 msgr2=0x7f82a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82a00a3c40 0x7f82a00a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82bc100a90 tx=0x7f82a800b260 comp rx=0 tx=0).stop 2024-04-17T07:57:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 msgr2=0x7f82bc199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc199130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e9a0 tx=0x7f82ac00ecb0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 shutdown_connections 2024-04-17T07:57:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f82a00a3c40 0x7f82a00a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82bc108980 0x7f82bc19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc102980 0x7f82bc199130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 --2- 172.21.15.46:0/4043131113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc103340 0x7f82bc199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:42.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 >> 172.21.15.46:0/4043131113 conn(0x7f82bc0fe4d0 msgr2=0x7f82bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:42.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 shutdown_connections 2024-04-17T07:57:42.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:42.949+0000 7f82bbe00700 1 -- 172.21.15.46:0/4043131113 wait complete. 2024-04-17T07:57:42.954 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:57:43.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:43 smithi046 ceph-mon[32038]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:43.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:43 smithi053 ceph-mon[36437]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:43.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:43 smithi062 ceph-mon[36507]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-04-17T07:57:41.563772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658150,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":785,"num_read_kb":510,"num_write":357,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181973,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380352,"kb_used_data":14480,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561472,"statfs":{"total":1151940427776,"available":1151550947328,"internally_reserved":0,"allocated":14827520,"data_stored":8185448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005651"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987034+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:57:22.987034+0000","last_peered":"2024-04-17T07:57:22.987034+0000","last_clean":"2024-04-17T07:57:22.987034+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:57:22.987034+0000","last_undegraded":"2024-04-17T07:57:22.987034+0000","last_fullsized":"2024-04-17T07:57:22.987034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301182+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:57:23.301182+0000","last_peered":"2024-04-17T07:57:23.301182+0000","last_clean":"2024-04-17T07:57:23.301182+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:57:23.301182+0000","last_undegraded":"2024-04-17T07:57:23.301182+0000","last_fullsized":"2024-04-17T07:57:23.301182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300823+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:57:23.300823+0000","last_peered":"2024-04-17T07:57:23.300823+0000","last_clean":"2024-04-17T07:57:23.300823+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:57:23.300823+0000","last_undegraded":"2024-04-17T07:57:23.300823+0000","last_fullsized":"2024-04-17T07:57:23.300823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987715+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:57:22.987715+0000","last_peered":"2024-04-17T07:57:22.987715+0000","last_clean":"2024-04-17T07:57:22.987715+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:57:22.987715+0000","last_undegraded":"2024-04-17T07:57:22.987715+0000","last_fullsized":"2024-04-17T07:57:22.987715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989568+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:57:22.989568+0000","last_peered":"2024-04-17T07:57:22.989568+0000","last_clean":"2024-04-17T07:57:22.989568+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:57:22.989568+0000","last_undegraded":"2024-04-17T07:57:22.989568+0000","last_fullsized":"2024-04-17T07:57:22.989568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990211+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:57:22.990211+0000","last_peered":"2024-04-17T07:57:22.990211+0000","last_clean":"2024-04-17T07:57:22.990211+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:57:22.990211+0000","last_undegraded":"2024-04-17T07:57:22.990211+0000","last_fullsized":"2024-04-17T07:57:22.990211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989436+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:57:22.989436+0000","last_peered":"2024-04-17T07:57:22.989436+0000","last_clean":"2024-04-17T07:57:22.989436+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:57:22.989436+0000","last_undegraded":"2024-04-17T07:57:22.989436+0000","last_fullsized":"2024-04-17T07:57:22.989436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981303+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:57:22.981303+0000","last_peered":"2024-04-17T07:57:22.981303+0000","last_clean":"2024-04-17T07:57:22.981303+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:57:22.981303+0000","last_undegraded":"2024-04-17T07:57:22.981303+0000","last_fullsized":"2024-04-17T07:57:22.981303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298370+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:57:23.298370+0000","last_peered":"2024-04-17T07:57:23.298370+0000","last_clean":"2024-04-17T07:57:23.298370+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:57:23.298370+0000","last_undegraded":"2024-04-17T07:57:23.298370+0000","last_fullsized":"2024-04-17T07:57:23.298370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991013+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:57:22.991013+0000","last_peered":"2024-04-17T07:57:22.991013+0000","last_clean":"2024-04-17T07:57:22.991013+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:57:22.991013+0000","last_undegraded":"2024-04-17T07:57:22.991013+0000","last_fullsized":"2024-04-17T07:57:22.991013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990203+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:57:22.990203+0000","last_peered":"2024-04-17T07:57:22.990203+0000","last_clean":"2024-04-17T07:57:22.990203+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:57:22.990203+0000","last_undegraded":"2024-04-17T07:57:22.990203+0000","last_fullsized":"2024-04-17T07:57:22.990203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987774+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:57:22.987774+0000","last_peered":"2024-04-17T07:57:22.987774+0000","last_clean":"2024-04-17T07:57:22.987774+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:57:22.987774+0000","last_undegraded":"2024-04-17T07:57:22.987774+0000","last_fullsized":"2024-04-17T07:57:22.987774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984322+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:57:22.984322+0000","last_peered":"2024-04-17T07:57:22.984322+0000","last_clean":"2024-04-17T07:57:22.984322+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:57:22.984322+0000","last_undegraded":"2024-04-17T07:57:22.984322+0000","last_fullsized":"2024-04-17T07:57:22.984322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300112+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:57:23.300112+0000","last_peered":"2024-04-17T07:57:23.300112+0000","last_clean":"2024-04-17T07:57:23.300112+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:57:23.300112+0000","last_undegraded":"2024-04-17T07:57:23.300112+0000","last_fullsized":"2024-04-17T07:57:23.300112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300251+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:57:23.300251+0000","last_peered":"2024-04-17T07:57:23.300251+0000","last_clean":"2024-04-17T07:57:23.300251+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:57:23.300251+0000","last_undegraded":"2024-04-17T07:57:23.300251+0000","last_fullsized":"2024-04-17T07:57:23.300251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297603+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:57:23.297603+0000","last_peered":"2024-04-17T07:57:23.297603+0000","last_clean":"2024-04-17T07:57:23.297603+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:57:23.297603+0000","last_undegraded":"2024-04-17T07:57:23.297603+0000","last_fullsized":"2024-04-17T07:57:23.297603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986826+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:57:22.986826+0000","last_peered":"2024-04-17T07:57:22.986826+0000","last_clean":"2024-04-17T07:57:22.986826+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:57:22.986826+0000","last_undegraded":"2024-04-17T07:57:22.986826+0000","last_fullsized":"2024-04-17T07:57:22.986826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299034+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:57:23.299034+0000","last_peered":"2024-04-17T07:57:23.299034+0000","last_clean":"2024-04-17T07:57:23.299034+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:57:23.299034+0000","last_undegraded":"2024-04-17T07:57:23.299034+0000","last_fullsized":"2024-04-17T07:57:23.299034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990468+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:57:22.990468+0000","last_peered":"2024-04-17T07:57:22.990468+0000","last_clean":"2024-04-17T07:57:22.990468+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:57:22.990468+0000","last_undegraded":"2024-04-17T07:57:22.990468+0000","last_fullsized":"2024-04-17T07:57:22.990468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990092+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:57:22.990092+0000","last_peered":"2024-04-17T07:57:22.990092+0000","last_clean":"2024-04-17T07:57:22.990092+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:57:22.990092+0000","last_undegraded":"2024-04-17T07:57:22.990092+0000","last_fullsized":"2024-04-17T07:57:22.990092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299637+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:57:23.299637+0000","last_peered":"2024-04-17T07:57:23.299637+0000","last_clean":"2024-04-17T07:57:23.299637+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:57:23.299637+0000","last_undegraded":"2024-04-17T07:57:23.299637+0000","last_fullsized":"2024-04-17T07:57:23.299637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298902+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:57:23.298902+0000","last_peered":"2024-04-17T07:57:23.298902+0000","last_clean":"2024-04-17T07:57:23.298902+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:57:23.298902+0000","last_undegraded":"2024-04-17T07:57:23.298902+0000","last_fullsized":"2024-04-17T07:57:23.298902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987051+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:57:22.987051+0000","last_peered":"2024-04-17T07:57:22.987051+0000","last_clean":"2024-04-17T07:57:22.987051+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:57:22.987051+0000","last_undegraded":"2024-04-17T07:57:22.987051+0000","last_fullsized":"2024-04-17T07:57:22.987051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984617+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:57:22.984617+0000","last_peered":"2024-04-17T07:57:22.984617+0000","last_clean":"2024-04-17T07:57:22.984617+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:57:22.984617+0000","last_undegraded":"2024-04-17T07:57:22.984617+0000","last_fullsized":"2024-04-17T07:57:22.984617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301222+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:57:23.301222+0000","last_peered":"2024-04-17T07:57:23.301222+0000","last_clean":"2024-04-17T07:57:23.301222+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:57:23.301222+0000","last_undegraded":"2024-04-17T07:57:23.301222+0000","last_fullsized":"2024-04-17T07:57:23.301222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987157+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:57:22.987157+0000","last_peered":"2024-04-17T07:57:22.987157+0000","last_clean":"2024-04-17T07:57:22.987157+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:57:22.987157+0000","last_undegraded":"2024-04-17T07:57:22.987157+0000","last_fullsized":"2024-04-17T07:57:22.987157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.996476+0000","last_change":"2024-04-17T07:57:22.996476+0000","last_active":"2024-04-17T07:57:22.996476+0000","last_peered":"2024-04-17T07:57:22.996476+0000","last_clean":"2024-04-17T07:57:22.996476+0000","last_became_active":"2024-04-17T07:57:22.991654+0000","last_became_peered":"2024-04-17T07:57:22.991654+0000","last_unstale":"2024-04-17T07:57:22.996476+0000","last_undegraded":"2024-04-17T07:57:22.996476+0000","last_fullsized":"2024-04-17T07:57:22.996476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:33.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298000+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:57:23.298000+0000","last_peered":"2024-04-17T07:57:23.298000+0000","last_clean":"2024-04-17T07:57:23.298000+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:57:23.298000+0000","last_undegraded":"2024-04-17T07:57:23.298000+0000","last_fullsized":"2024-04-17T07:57:23.298000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987060+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:57:22.987060+0000","last_peered":"2024-04-17T07:57:22.987060+0000","last_clean":"2024-04-17T07:57:22.987060+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:57:22.987060+0000","last_undegraded":"2024-04-17T07:57:22.987060+0000","last_fullsized":"2024-04-17T07:57:22.987060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990006+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:57:22.990006+0000","last_peered":"2024-04-17T07:57:22.990006+0000","last_clean":"2024-04-17T07:57:22.990006+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:57:22.990006+0000","last_undegraded":"2024-04-17T07:57:22.990006+0000","last_fullsized":"2024-04-17T07:57:22.990006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981957+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:57:22.981957+0000","last_peered":"2024-04-17T07:57:22.981957+0000","last_clean":"2024-04-17T07:57:22.981957+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:57:22.981957+0000","last_undegraded":"2024-04-17T07:57:22.981957+0000","last_fullsized":"2024-04-17T07:57:22.981957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990345+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:57:22.990345+0000","last_peered":"2024-04-17T07:57:22.990345+0000","last_clean":"2024-04-17T07:57:22.990345+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:57:22.990345+0000","last_undegraded":"2024-04-17T07:57:22.990345+0000","last_fullsized":"2024-04-17T07:57:22.990345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:39.706269+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:57:39.706269+0000","last_peered":"2024-04-17T07:57:39.706269+0000","last_clean":"2024-04-17T07:57:39.706269+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:57:39.706269+0000","last_undegraded":"2024-04-17T07:57:39.706269+0000","last_fullsized":"2024-04-17T07:57:39.706269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989890+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:57:22.989890+0000","last_peered":"2024-04-17T07:57:22.989890+0000","last_clean":"2024-04-17T07:57:22.989890+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:57:22.989890+0000","last_undegraded":"2024-04-17T07:57:22.989890+0000","last_fullsized":"2024-04-17T07:57:22.989890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989695+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:57:22.989695+0000","last_peered":"2024-04-17T07:57:22.989695+0000","last_clean":"2024-04-17T07:57:22.989695+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:57:22.989695+0000","last_undegraded":"2024-04-17T07:57:22.989695+0000","last_fullsized":"2024-04-17T07:57:22.989695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990075+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:57:22.990075+0000","last_peered":"2024-04-17T07:57:22.990075+0000","last_clean":"2024-04-17T07:57:22.990075+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:57:22.990075+0000","last_undegraded":"2024-04-17T07:57:22.990075+0000","last_fullsized":"2024-04-17T07:57:22.990075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981716+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:57:22.981716+0000","last_peered":"2024-04-17T07:57:22.981716+0000","last_clean":"2024-04-17T07:57:22.981716+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:57:22.981716+0000","last_undegraded":"2024-04-17T07:57:22.981716+0000","last_fullsized":"2024-04-17T07:57:22.981716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986902+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:57:22.986902+0000","last_peered":"2024-04-17T07:57:22.986902+0000","last_clean":"2024-04-17T07:57:22.986902+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:57:22.986902+0000","last_undegraded":"2024-04-17T07:57:22.986902+0000","last_fullsized":"2024-04-17T07:57:22.986902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982797+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:57:22.982797+0000","last_peered":"2024-04-17T07:57:22.982797+0000","last_clean":"2024-04-17T07:57:22.982797+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:57:22.982797+0000","last_undegraded":"2024-04-17T07:57:22.982797+0000","last_fullsized":"2024-04-17T07:57:22.982797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.980936+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:57:22.980936+0000","last_peered":"2024-04-17T07:57:22.980936+0000","last_clean":"2024-04-17T07:57:22.980936+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:57:22.980936+0000","last_undegraded":"2024-04-17T07:57:22.980936+0000","last_fullsized":"2024-04-17T07:57:22.980936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301002+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:57:23.301002+0000","last_peered":"2024-04-17T07:57:23.301002+0000","last_clean":"2024-04-17T07:57:23.301002+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:57:23.301002+0000","last_undegraded":"2024-04-17T07:57:23.301002+0000","last_fullsized":"2024-04-17T07:57:23.301002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981394+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:57:22.981394+0000","last_peered":"2024-04-17T07:57:22.981394+0000","last_clean":"2024-04-17T07:57:22.981394+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:57:22.981394+0000","last_undegraded":"2024-04-17T07:57:22.981394+0000","last_fullsized":"2024-04-17T07:57:22.981394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299000+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:57:23.299000+0000","last_peered":"2024-04-17T07:57:23.299000+0000","last_clean":"2024-04-17T07:57:23.299000+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:57:23.299000+0000","last_undegraded":"2024-04-17T07:57:23.299000+0000","last_fullsized":"2024-04-17T07:57:23.299000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981197+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:57:22.981197+0000","last_peered":"2024-04-17T07:57:22.981197+0000","last_clean":"2024-04-17T07:57:22.981197+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:57:22.981197+0000","last_undegraded":"2024-04-17T07:57:22.981197+0000","last_fullsized":"2024-04-17T07:57:22.981197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":138,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:35.699122+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:57:35.699122+0000","last_peered":"2024-04-17T07:57:35.699122+0000","last_clean":"2024-04-17T07:57:35.699122+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:57:35.699122+0000","last_undegraded":"2024-04-17T07:57:35.699122+0000","last_fullsized":"2024-04-17T07:57:35.699122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298379+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:57:23.298379+0000","last_peered":"2024-04-17T07:57:23.298379+0000","last_clean":"2024-04-17T07:57:23.298379+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:57:23.298379+0000","last_undegraded":"2024-04-17T07:57:23.298379+0000","last_fullsized":"2024-04-17T07:57:23.298379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299142+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:57:23.299142+0000","last_peered":"2024-04-17T07:57:23.299142+0000","last_clean":"2024-04-17T07:57:23.299142+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:57:23.299142+0000","last_undegraded":"2024-04-17T07:57:23.299142+0000","last_fullsized":"2024-04-17T07:57:23.299142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989795+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:57:22.989795+0000","last_peered":"2024-04-17T07:57:22.989795+0000","last_clean":"2024-04-17T07:57:22.989795+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:57:22.989795+0000","last_undegraded":"2024-04-17T07:57:22.989795+0000","last_fullsized":"2024-04-17T07:57:22.989795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301097+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:57:23.301097+0000","last_peered":"2024-04-17T07:57:23.301097+0000","last_clean":"2024-04-17T07:57:23.301097+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:57:23.301097+0000","last_undegraded":"2024-04-17T07:57:23.301097+0000","last_fullsized":"2024-04-17T07:57:23.301097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987275+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:57:22.987275+0000","last_peered":"2024-04-17T07:57:22.987275+0000","last_clean":"2024-04-17T07:57:22.987275+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:57:22.987275+0000","last_undegraded":"2024-04-17T07:57:22.987275+0000","last_fullsized":"2024-04-17T07:57:22.987275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981928+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:57:22.981928+0000","last_peered":"2024-04-17T07:57:22.981928+0000","last_clean":"2024-04-17T07:57:22.981928+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:57:22.981928+0000","last_undegraded":"2024-04-17T07:57:22.981928+0000","last_fullsized":"2024-04-17T07:57:22.981928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298113+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:57:23.298113+0000","last_peered":"2024-04-17T07:57:23.298113+0000","last_clean":"2024-04-17T07:57:23.298113+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:57:23.298113+0000","last_undegraded":"2024-04-17T07:57:23.298113+0000","last_fullsized":"2024-04-17T07:57:23.298113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298615+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:57:23.298615+0000","last_peered":"2024-04-17T07:57:23.298615+0000","last_clean":"2024-04-17T07:57:23.298615+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:57:23.298615+0000","last_undegraded":"2024-04-17T07:57:23.298615+0000","last_fullsized":"2024-04-17T07:57:23.298615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":281,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:36.700821+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:57:36.700821+0000","last_peered":"2024-04-17T07:57:36.700821+0000","last_clean":"2024-04-17T07:57:36.700821+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:57:36.700821+0000","last_undegraded":"2024-04-17T07:57:36.700821+0000","last_fullsized":"2024-04-17T07:57:36.700821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":215,"num_read_kb":170,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991075+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:57:22.991075+0000","last_peered":"2024-04-17T07:57:22.991075+0000","last_clean":"2024-04-17T07:57:22.991075+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:57:22.991075+0000","last_undegraded":"2024-04-17T07:57:22.991075+0000","last_fullsized":"2024-04-17T07:57:22.991075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982168+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:57:22.982168+0000","last_peered":"2024-04-17T07:57:22.982168+0000","last_clean":"2024-04-17T07:57:22.982168+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:57:22.982168+0000","last_undegraded":"2024-04-17T07:57:22.982168+0000","last_fullsized":"2024-04-17T07:57:22.982168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989392+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:57:22.989392+0000","last_peered":"2024-04-17T07:57:22.989392+0000","last_clean":"2024-04-17T07:57:22.989392+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:57:22.989392+0000","last_undegraded":"2024-04-17T07:57:22.989392+0000","last_fullsized":"2024-04-17T07:57:22.989392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983358+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:57:22.983358+0000","last_peered":"2024-04-17T07:57:22.983358+0000","last_clean":"2024-04-17T07:57:22.983358+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:57:22.983358+0000","last_undegraded":"2024-04-17T07:57:22.983358+0000","last_fullsized":"2024-04-17T07:57:22.983358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986309+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:57:22.986309+0000","last_peered":"2024-04-17T07:57:22.986309+0000","last_clean":"2024-04-17T07:57:22.986309+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:57:22.986309+0000","last_undegraded":"2024-04-17T07:57:22.986309+0000","last_fullsized":"2024-04-17T07:57:22.986309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981818+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:57:22.981818+0000","last_peered":"2024-04-17T07:57:22.981818+0000","last_clean":"2024-04-17T07:57:22.981818+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:57:22.981818+0000","last_undegraded":"2024-04-17T07:57:22.981818+0000","last_fullsized":"2024-04-17T07:57:22.981818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989572+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:57:22.989572+0000","last_peered":"2024-04-17T07:57:22.989572+0000","last_clean":"2024-04-17T07:57:22.989572+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:57:22.989572+0000","last_undegraded":"2024-04-17T07:57:22.989572+0000","last_fullsized":"2024-04-17T07:57:22.989572+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299360+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:57:23.299360+0000","last_peered":"2024-04-17T07:57:23.299360+0000","last_clean":"2024-04-17T07:57:23.299360+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:57:23.299360+0000","last_undegraded":"2024-04-17T07:57:23.299360+0000","last_fullsized":"2024-04-17T07:57:23.299360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:40.034404+0000","last_change":"2024-04-17T07:57:22.995880+0000","last_active":"2024-04-17T07:57:40.034404+0000","last_peered":"2024-04-17T07:57:40.034404+0000","last_clean":"2024-04-17T07:57:40.034404+0000","last_became_active":"2024-04-17T07:57:22.991967+0000","last_became_peered":"2024-04-17T07:57:22.991967+0000","last_unstale":"2024-04-17T07:57:40.034404+0000","last_undegraded":"2024-04-17T07:57:40.034404+0000","last_fullsized":"2024-04-17T07:57:40.034404+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:27.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989483+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:57:22.989483+0000","last_peered":"2024-04-17T07:57:22.989483+0000","last_clean":"2024-04-17T07:57:22.989483+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:57:22.989483+0000","last_undegraded":"2024-04-17T07:57:22.989483+0000","last_fullsized":"2024-04-17T07:57:22.989483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984496+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:57:22.984496+0000","last_peered":"2024-04-17T07:57:22.984496+0000","last_clean":"2024-04-17T07:57:22.984496+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:57:22.984496+0000","last_undegraded":"2024-04-17T07:57:22.984496+0000","last_fullsized":"2024-04-17T07:57:22.984496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989772+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:57:22.989772+0000","last_peered":"2024-04-17T07:57:22.989772+0000","last_clean":"2024-04-17T07:57:22.989772+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:57:22.989772+0000","last_undegraded":"2024-04-17T07:57:22.989772+0000","last_fullsized":"2024-04-17T07:57:22.989772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981395+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:57:22.981395+0000","last_peered":"2024-04-17T07:57:22.981395+0000","last_clean":"2024-04-17T07:57:22.981395+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:57:22.981395+0000","last_undegraded":"2024-04-17T07:57:22.981395+0000","last_fullsized":"2024-04-17T07:57:22.981395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981392+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:57:22.981392+0000","last_peered":"2024-04-17T07:57:22.981392+0000","last_clean":"2024-04-17T07:57:22.981392+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:57:22.981392+0000","last_undegraded":"2024-04-17T07:57:22.981392+0000","last_fullsized":"2024-04-17T07:57:22.981392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991206+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:57:22.991206+0000","last_peered":"2024-04-17T07:57:22.991206+0000","last_clean":"2024-04-17T07:57:22.991206+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:57:22.991206+0000","last_undegraded":"2024-04-17T07:57:22.991206+0000","last_fullsized":"2024-04-17T07:57:22.991206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986817+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:57:22.986817+0000","last_peered":"2024-04-17T07:57:22.986817+0000","last_clean":"2024-04-17T07:57:22.986817+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:57:22.986817+0000","last_undegraded":"2024-04-17T07:57:22.986817+0000","last_fullsized":"2024-04-17T07:57:22.986817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989693+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:57:22.989693+0000","last_peered":"2024-04-17T07:57:22.989693+0000","last_clean":"2024-04-17T07:57:22.989693+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:57:22.989693+0000","last_undegraded":"2024-04-17T07:57:22.989693+0000","last_fullsized":"2024-04-17T07:57:22.989693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299647+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:57:23.299647+0000","last_peered":"2024-04-17T07:57:23.299647+0000","last_clean":"2024-04-17T07:57:23.299647+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:57:23.299647+0000","last_undegraded":"2024-04-17T07:57:23.299647+0000","last_fullsized":"2024-04-17T07:57:23.299647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982610+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:57:22.982610+0000","last_peered":"2024-04-17T07:57:22.982610+0000","last_clean":"2024-04-17T07:57:22.982610+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:57:22.982610+0000","last_undegraded":"2024-04-17T07:57:22.982610+0000","last_fullsized":"2024-04-17T07:57:22.982610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989416+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:57:22.989416+0000","last_peered":"2024-04-17T07:57:22.989416+0000","last_clean":"2024-04-17T07:57:22.989416+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:57:22.989416+0000","last_undegraded":"2024-04-17T07:57:22.989416+0000","last_fullsized":"2024-04-17T07:57:22.989416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990564+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:57:22.990564+0000","last_peered":"2024-04-17T07:57:22.990564+0000","last_clean":"2024-04-17T07:57:22.990564+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:57:22.990564+0000","last_undegraded":"2024-04-17T07:57:22.990564+0000","last_fullsized":"2024-04-17T07:57:22.990564+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.985794+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:57:22.985794+0000","last_peered":"2024-04-17T07:57:22.985794+0000","last_clean":"2024-04-17T07:57:22.985794+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:57:22.985794+0000","last_undegraded":"2024-04-17T07:57:22.985794+0000","last_fullsized":"2024-04-17T07:57:22.985794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:40.033377+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:57:40.033377+0000","last_peered":"2024-04-17T07:57:40.033377+0000","last_clean":"2024-04-17T07:57:40.033377+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:57:40.033377+0000","last_undegraded":"2024-04-17T07:57:40.033377+0000","last_fullsized":"2024-04-17T07:57:40.033377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989218+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:57:22.989218+0000","last_peered":"2024-04-17T07:57:22.989218+0000","last_clean":"2024-04-17T07:57:22.989218+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:57:22.989218+0000","last_undegraded":"2024-04-17T07:57:22.989218+0000","last_fullsized":"2024-04-17T07:57:22.989218+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982675+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:57:22.982675+0000","last_peered":"2024-04-17T07:57:22.982675+0000","last_clean":"2024-04-17T07:57:22.982675+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:57:22.982675+0000","last_undegraded":"2024-04-17T07:57:22.982675+0000","last_fullsized":"2024-04-17T07:57:22.982675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987305+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:57:22.987305+0000","last_peered":"2024-04-17T07:57:22.987305+0000","last_clean":"2024-04-17T07:57:22.987305+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:57:22.987305+0000","last_undegraded":"2024-04-17T07:57:22.987305+0000","last_fullsized":"2024-04-17T07:57:22.987305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989188+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:57:22.989188+0000","last_peered":"2024-04-17T07:57:22.989188+0000","last_clean":"2024-04-17T07:57:22.989188+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:57:22.989188+0000","last_undegraded":"2024-04-17T07:57:22.989188+0000","last_fullsized":"2024-04-17T07:57:22.989188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982074+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:57:22.982074+0000","last_peered":"2024-04-17T07:57:22.982074+0000","last_clean":"2024-04-17T07:57:22.982074+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:57:22.982074+0000","last_undegraded":"2024-04-17T07:57:22.982074+0000","last_fullsized":"2024-04-17T07:57:22.982074+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982417+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:57:22.982417+0000","last_peered":"2024-04-17T07:57:22.982417+0000","last_clean":"2024-04-17T07:57:22.982417+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:57:22.982417+0000","last_undegraded":"2024-04-17T07:57:22.982417+0000","last_fullsized":"2024-04-17T07:57:22.982417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989408+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:57:22.989408+0000","last_peered":"2024-04-17T07:57:22.989408+0000","last_clean":"2024-04-17T07:57:22.989408+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:57:22.989408+0000","last_undegraded":"2024-04-17T07:57:22.989408+0000","last_fullsized":"2024-04-17T07:57:22.989408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986358+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:57:22.986358+0000","last_peered":"2024-04-17T07:57:22.986358+0000","last_clean":"2024-04-17T07:57:22.986358+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:57:22.986358+0000","last_undegraded":"2024-04-17T07:57:22.986358+0000","last_fullsized":"2024-04-17T07:57:22.986358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983466+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:57:22.983466+0000","last_peered":"2024-04-17T07:57:22.983466+0000","last_clean":"2024-04-17T07:57:22.983466+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:57:22.983466+0000","last_undegraded":"2024-04-17T07:57:22.983466+0000","last_fullsized":"2024-04-17T07:57:22.983466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981116+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:57:22.981116+0000","last_peered":"2024-04-17T07:57:22.981116+0000","last_clean":"2024-04-17T07:57:22.981116+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:57:22.981116+0000","last_undegraded":"2024-04-17T07:57:22.981116+0000","last_fullsized":"2024-04-17T07:57:22.981116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299253+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:57:23.299253+0000","last_peered":"2024-04-17T07:57:23.299253+0000","last_clean":"2024-04-17T07:57:23.299253+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:57:23.299253+0000","last_undegraded":"2024-04-17T07:57:23.299253+0000","last_fullsized":"2024-04-17T07:57:23.299253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984249+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:57:22.984249+0000","last_peered":"2024-04-17T07:57:22.984249+0000","last_clean":"2024-04-17T07:57:22.984249+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:57:22.984249+0000","last_undegraded":"2024-04-17T07:57:22.984249+0000","last_fullsized":"2024-04-17T07:57:22.984249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989826+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:57:22.989826+0000","last_peered":"2024-04-17T07:57:22.989826+0000","last_clean":"2024-04-17T07:57:22.989826+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:57:22.989826+0000","last_undegraded":"2024-04-17T07:57:22.989826+0000","last_fullsized":"2024-04-17T07:57:22.989826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987958+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:57:22.987958+0000","last_peered":"2024-04-17T07:57:22.987958+0000","last_clean":"2024-04-17T07:57:22.987958+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:57:22.987958+0000","last_undegraded":"2024-04-17T07:57:22.987958+0000","last_fullsized":"2024-04-17T07:57:22.987958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301351+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:57:23.301351+0000","last_peered":"2024-04-17T07:57:23.301351+0000","last_clean":"2024-04-17T07:57:23.301351+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:57:23.301351+0000","last_undegraded":"2024-04-17T07:57:23.301351+0000","last_fullsized":"2024-04-17T07:57:23.301351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987589+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:57:22.987589+0000","last_peered":"2024-04-17T07:57:22.987589+0000","last_clean":"2024-04-17T07:57:22.987589+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:57:22.987589+0000","last_undegraded":"2024-04-17T07:57:22.987589+0000","last_fullsized":"2024-04-17T07:57:22.987589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297743+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:57:23.297743+0000","last_peered":"2024-04-17T07:57:23.297743+0000","last_clean":"2024-04-17T07:57:23.297743+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:57:23.297743+0000","last_undegraded":"2024-04-17T07:57:23.297743+0000","last_fullsized":"2024-04-17T07:57:23.297743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989106+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:57:22.989106+0000","last_peered":"2024-04-17T07:57:22.989106+0000","last_clean":"2024-04-17T07:57:22.989106+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:57:22.989106+0000","last_undegraded":"2024-04-17T07:57:22.989106+0000","last_fullsized":"2024-04-17T07:57:22.989106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299526+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:57:23.299526+0000","last_peered":"2024-04-17T07:57:23.299526+0000","last_clean":"2024-04-17T07:57:23.299526+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:57:23.299526+0000","last_undegraded":"2024-04-17T07:57:23.299526+0000","last_fullsized":"2024-04-17T07:57:23.299526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981676+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:57:22.981676+0000","last_peered":"2024-04-17T07:57:22.981676+0000","last_clean":"2024-04-17T07:57:22.981676+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:57:22.981676+0000","last_undegraded":"2024-04-17T07:57:22.981676+0000","last_fullsized":"2024-04-17T07:57:22.981676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986346+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:57:22.986346+0000","last_peered":"2024-04-17T07:57:22.986346+0000","last_clean":"2024-04-17T07:57:22.986346+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:57:22.986346+0000","last_undegraded":"2024-04-17T07:57:22.986346+0000","last_fullsized":"2024-04-17T07:57:22.986346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987327+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:57:22.987327+0000","last_peered":"2024-04-17T07:57:22.987327+0000","last_clean":"2024-04-17T07:57:22.987327+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:57:22.987327+0000","last_undegraded":"2024-04-17T07:57:22.987327+0000","last_fullsized":"2024-04-17T07:57:22.987327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990346+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:57:22.990346+0000","last_peered":"2024-04-17T07:57:22.990346+0000","last_clean":"2024-04-17T07:57:22.990346+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:57:22.990346+0000","last_undegraded":"2024-04-17T07:57:22.990346+0000","last_fullsized":"2024-04-17T07:57:22.990346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983499+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:57:22.983499+0000","last_peered":"2024-04-17T07:57:22.983499+0000","last_clean":"2024-04-17T07:57:22.983499+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:57:22.983499+0000","last_undegraded":"2024-04-17T07:57:22.983499+0000","last_fullsized":"2024-04-17T07:57:22.983499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987653+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:57:22.987653+0000","last_peered":"2024-04-17T07:57:22.987653+0000","last_clean":"2024-04-17T07:57:22.987653+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:57:22.987653+0000","last_undegraded":"2024-04-17T07:57:22.987653+0000","last_fullsized":"2024-04-17T07:57:22.987653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989692+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:57:22.989692+0000","last_peered":"2024-04-17T07:57:22.989692+0000","last_clean":"2024-04-17T07:57:22.989692+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:57:22.989692+0000","last_undegraded":"2024-04-17T07:57:22.989692+0000","last_fullsized":"2024-04-17T07:57:22.989692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989548+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:57:22.989548+0000","last_peered":"2024-04-17T07:57:22.989548+0000","last_clean":"2024-04-17T07:57:22.989548+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:57:22.989548+0000","last_undegraded":"2024-04-17T07:57:22.989548+0000","last_fullsized":"2024-04-17T07:57:22.989548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297780+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:57:23.297780+0000","last_peered":"2024-04-17T07:57:23.297780+0000","last_clean":"2024-04-17T07:57:23.297780+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:57:23.297780+0000","last_undegraded":"2024-04-17T07:57:23.297780+0000","last_fullsized":"2024-04-17T07:57:23.297780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989968+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:57:22.989968+0000","last_peered":"2024-04-17T07:57:22.989968+0000","last_clean":"2024-04-17T07:57:22.989968+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:57:22.989968+0000","last_undegraded":"2024-04-17T07:57:22.989968+0000","last_fullsized":"2024-04-17T07:57:22.989968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990328+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:57:22.990328+0000","last_peered":"2024-04-17T07:57:22.990328+0000","last_clean":"2024-04-17T07:57:22.990328+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:57:22.990328+0000","last_undegraded":"2024-04-17T07:57:22.990328+0000","last_fullsized":"2024-04-17T07:57:22.990328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299788+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:57:23.299788+0000","last_peered":"2024-04-17T07:57:23.299788+0000","last_clean":"2024-04-17T07:57:23.299788+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:57:23.299788+0000","last_undegraded":"2024-04-17T07:57:23.299788+0000","last_fullsized":"2024-04-17T07:57:23.299788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982144+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:57:22.982144+0000","last_peered":"2024-04-17T07:57:22.982144+0000","last_clean":"2024-04-17T07:57:22.982144+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:57:22.982144+0000","last_undegraded":"2024-04-17T07:57:22.982144+0000","last_fullsized":"2024-04-17T07:57:22.982144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989766+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:57:22.989766+0000","last_peered":"2024-04-17T07:57:22.989766+0000","last_clean":"2024-04-17T07:57:22.989766+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:57:22.989766+0000","last_undegraded":"2024-04-17T07:57:22.989766+0000","last_fullsized":"2024-04-17T07:57:22.989766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981278+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:57:22.981278+0000","last_peered":"2024-04-17T07:57:22.981278+0000","last_clean":"2024-04-17T07:57:22.981278+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:57:22.981278+0000","last_undegraded":"2024-04-17T07:57:22.981278+0000","last_fullsized":"2024-04-17T07:57:22.981278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984913+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:57:22.984913+0000","last_peered":"2024-04-17T07:57:22.984913+0000","last_clean":"2024-04-17T07:57:22.984913+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:57:22.984913+0000","last_undegraded":"2024-04-17T07:57:22.984913+0000","last_fullsized":"2024-04-17T07:57:22.984913+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989884+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:57:22.989884+0000","last_peered":"2024-04-17T07:57:22.989884+0000","last_clean":"2024-04-17T07:57:22.989884+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:57:22.989884+0000","last_undegraded":"2024-04-17T07:57:22.989884+0000","last_fullsized":"2024-04-17T07:57:22.989884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986499+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:57:22.986499+0000","last_peered":"2024-04-17T07:57:22.986499+0000","last_clean":"2024-04-17T07:57:22.986499+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:57:22.986499+0000","last_undegraded":"2024-04-17T07:57:22.986499+0000","last_fullsized":"2024-04-17T07:57:22.986499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990103+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:57:22.990103+0000","last_peered":"2024-04-17T07:57:22.990103+0000","last_clean":"2024-04-17T07:57:22.990103+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:57:22.990103+0000","last_undegraded":"2024-04-17T07:57:22.990103+0000","last_fullsized":"2024-04-17T07:57:22.990103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990216+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:57:22.990216+0000","last_peered":"2024-04-17T07:57:22.990216+0000","last_clean":"2024-04-17T07:57:22.990216+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:57:22.990216+0000","last_undegraded":"2024-04-17T07:57:22.990216+0000","last_fullsized":"2024-04-17T07:57:22.990216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300391+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:57:23.300391+0000","last_peered":"2024-04-17T07:57:23.300391+0000","last_clean":"2024-04-17T07:57:23.300391+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:57:23.300391+0000","last_undegraded":"2024-04-17T07:57:23.300391+0000","last_fullsized":"2024-04-17T07:57:23.300391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983162+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:57:22.983162+0000","last_peered":"2024-04-17T07:57:22.983162+0000","last_clean":"2024-04-17T07:57:22.983162+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:57:22.983162+0000","last_undegraded":"2024-04-17T07:57:22.983162+0000","last_fullsized":"2024-04-17T07:57:22.983162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986726+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:57:22.986726+0000","last_peered":"2024-04-17T07:57:22.986726+0000","last_clean":"2024-04-17T07:57:22.986726+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:57:22.986726+0000","last_undegraded":"2024-04-17T07:57:22.986726+0000","last_fullsized":"2024-04-17T07:57:22.986726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990569+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:57:22.990569+0000","last_peered":"2024-04-17T07:57:22.990569+0000","last_clean":"2024-04-17T07:57:22.990569+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:57:22.990569+0000","last_undegraded":"2024-04-17T07:57:22.990569+0000","last_fullsized":"2024-04-17T07:57:22.990569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984737+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:57:22.984737+0000","last_peered":"2024-04-17T07:57:22.984737+0000","last_clean":"2024-04-17T07:57:22.984737+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:57:22.984737+0000","last_undegraded":"2024-04-17T07:57:22.984737+0000","last_fullsized":"2024-04-17T07:57:22.984737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981744+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:57:22.981744+0000","last_peered":"2024-04-17T07:57:22.981744+0000","last_clean":"2024-04-17T07:57:22.981744+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:57:22.981744+0000","last_undegraded":"2024-04-17T07:57:22.981744+0000","last_fullsized":"2024-04-17T07:57:22.981744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981814+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:57:22.981814+0000","last_peered":"2024-04-17T07:57:22.981814+0000","last_clean":"2024-04-17T07:57:22.981814+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:57:22.981814+0000","last_undegraded":"2024-04-17T07:57:22.981814+0000","last_fullsized":"2024-04-17T07:57:22.981814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298052+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:57:23.298052+0000","last_peered":"2024-04-17T07:57:23.298052+0000","last_clean":"2024-04-17T07:57:23.298052+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:57:23.298052+0000","last_undegraded":"2024-04-17T07:57:23.298052+0000","last_fullsized":"2024-04-17T07:57:23.298052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984677+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:57:22.984677+0000","last_peered":"2024-04-17T07:57:22.984677+0000","last_clean":"2024-04-17T07:57:22.984677+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:57:22.984677+0000","last_undegraded":"2024-04-17T07:57:22.984677+0000","last_fullsized":"2024-04-17T07:57:22.984677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986489+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:57:22.986489+0000","last_peered":"2024-04-17T07:57:22.986489+0000","last_clean":"2024-04-17T07:57:22.986489+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:57:22.986489+0000","last_undegraded":"2024-04-17T07:57:22.986489+0000","last_fullsized":"2024-04-17T07:57:22.986489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67548,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":401,"num_write":174,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1748992,"data_stored":1170263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":523921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":523841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1115207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":577785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":579667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1107287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:57:43.850 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-17T07:57:43.850 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T07:57:43.850 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-17T07:57:43.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph health --format=json 2024-04-17T07:57:44.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:44 smithi046 ceph-mon[32038]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:44.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:44 smithi053 ceph-mon[36437]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:44.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:44 smithi062 ceph-mon[36507]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:57:45.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:45 smithi046 ceph-mon[32038]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:45.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:45 smithi053 ceph-mon[36437]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:45.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:45 smithi062 ceph-mon[36507]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 -- 172.21.15.46:0/400227866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a01092e0 msgr2=0x7fd7a01096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:46.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/400227866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a01092e0 0x7fd7a01096c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd7940099c0 tx=0x7fd794009cd0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 -- 172.21.15.46:0/400227866 shutdown_connections 2024-04-17T07:57:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/400227866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a010e3c0 0x7fd7a0115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/400227866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a01092e0 0x7fd7a01096c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/400227866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a0109c90 0x7fd7a010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 -- 172.21.15.46:0/400227866 >> 172.21.15.46:0/400227866 conn(0x7fd7a0076220 msgr2=0x7fd7a0107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 -- 172.21.15.46:0/400227866 shutdown_connections 2024-04-17T07:57:46.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.408+0000 7fd7a7e00700 1 -- 172.21.15.46:0/400227866 wait complete. 2024-04-17T07:57:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.409+0000 7fd7a7e00700 1 Processor -- start 2024-04-17T07:57:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.409+0000 7fd7a7e00700 1 -- start start 2024-04-17T07:57:46.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a7e00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a01092e0 0x7fd7a01a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a7e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a0109c90 0x7fd7a01a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a010e3c0 0x7fd7a019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a7e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01193a0 con 0x7fd7a010e3c0 2024-04-17T07:57:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a7e00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0119220 con 0x7fd7a01092e0 2024-04-17T07:57:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a7e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0119520 con 0x7fd7a0109c90 2024-04-17T07:57:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a01092e0 0x7fd7a01a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a5e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a010e3c0 0x7fd7a019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a5400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a01092e0 0x7fd7a01a5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56928/0 (socket says 172.21.15.46:56928) 2024-04-17T07:57:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a4a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a0109c90 0x7fd7a01a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.410+0000 7fd7a5400700 1 -- 172.21.15.46:0/2526547751 learned_addr learned my addr 172.21.15.46:0/2526547751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a5400700 1 -- 172.21.15.46:0/2526547751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a0109c90 msgr2=0x7fd7a01a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a5400700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a0109c90 0x7fd7a01a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a5400700 1 -- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a010e3c0 msgr2=0x7fd7a019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a5400700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a010e3c0 0x7fd7a019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a5400700 1 -- 172.21.15.46:0/2526547751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a019ff40 con 0x7fd7a01092e0 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a4a00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a0109c90 0x7fd7a01a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a5400700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a01092e0 0x7fd7a01a5590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd794006010 tx=0x7fd794003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd79e000700 1 -- 172.21.15.46:0/2526547751 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794003680 con 0x7fd7a01092e0 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd79e000700 1 -- 172.21.15.46:0/2526547751 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7940055a0 con 0x7fd7a01092e0 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd79e000700 1 -- 172.21.15.46:0/2526547751 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79400fe50 con 0x7fd7a01092e0 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.411+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a01a01d0 con 0x7fd7a01092e0 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.412+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01ac340 con 0x7fd7a01092e0 2024-04-17T07:57:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.413+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a010a7c0 con 0x7fd7a01092e0 2024-04-17T07:57:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.415+0000 7fd79e000700 1 -- 172.21.15.46:0/2526547751 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd79400f580 con 0x7fd7a01092e0 2024-04-17T07:57:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.416+0000 7fd79e000700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7880a4000 0x7fd7880a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.416+0000 7fd79e000700 1 -- 172.21.15.46:0/2526547751 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd7940d13e0 con 0x7fd7a01092e0 2024-04-17T07:57:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.417+0000 7fd7a4a00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7880a4000 0x7fd7880a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.417+0000 7fd7a4a00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7880a4000 0x7fd7880a64c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd7a01a0e20 tx=0x7fd78c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:46.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.419+0000 7fd79e000700 1 -- 172.21.15.46:0/2526547751 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd794013070 con 0x7fd7a01092e0 2024-04-17T07:57:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.709+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd7a01a0b90 con 0x7fd7a01092e0 2024-04-17T07:57:46.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.709+0000 7fd79e000700 1 -- 172.21.15.46:0/2526547751 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd79409da60 con 0x7fd7a01092e0 2024-04-17T07:57:46.710 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:46.710 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T07:57:46.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.712+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7880a4000 msgr2=0x7fd7880a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.712+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7880a4000 0x7fd7880a64c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd7a01a0e20 tx=0x7fd78c009260 comp rx=0 tx=0).stop 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.712+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a01092e0 msgr2=0x7fd7a01a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.712+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a01092e0 0x7fd7a01a5590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd794006010 tx=0x7fd794003a90 comp rx=0 tx=0).stop 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 shutdown_connections 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7880a4000 0x7fd7880a64c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a010e3c0 0x7fd7a019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7a01092e0 0x7fd7a01a5590 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 --2- 172.21.15.46:0/2526547751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a0109c90 0x7fd7a01a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 >> 172.21.15.46:0/2526547751 conn(0x7fd7a0076220 msgr2=0x7fd7a0077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 shutdown_connections 2024-04-17T07:57:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:46.713+0000 7fd7a7e00700 1 -- 172.21.15.46:0/2526547751 wait complete. 2024-04-17T07:57:47.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:47 smithi046 ceph-mon[32038]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:47.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:47 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2526547751' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T07:57:47.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:47 smithi053 ceph-mon[36437]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:47.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:47 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2526547751' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T07:57:47.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:47 smithi062 ceph-mon[36507]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:47.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:47 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2526547751' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T07:57:47.576 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-17T07:57:47.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 -- 172.21.15.46:0/1025588907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8109ec0 msgr2=0x7fa6e810c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 --2- 172.21.15.46:0/1025588907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8109ec0 0x7fa6e810c360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009ca0 tx=0x7fa6e002c180 comp rx=0 tx=0).stop 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 -- 172.21.15.46:0/1025588907 shutdown_connections 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 --2- 172.21.15.46:0/1025588907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8109ec0 0x7fa6e810c360 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 --2- 172.21.15.46:0/1025588907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8074960 0x7fa6e8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 --2- 172.21.15.46:0/1025588907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6e8075f40 0x7fa6e8074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 -- 172.21.15.46:0/1025588907 >> 172.21.15.46:0/1025588907 conn(0x7fa6e80fd580 msgr2=0x7fa6e80ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 -- 172.21.15.46:0/1025588907 shutdown_connections 2024-04-17T07:57:47.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.819+0000 7fa6eec00640 1 -- 172.21.15.46:0/1025588907 wait complete. 2024-04-17T07:57:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.820+0000 7fa6eec00640 1 Processor -- start 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6eec00640 1 -- start start 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6eec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8074960 0x7fa6e819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6eec00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8075f40 0x7fa6e819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6eec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6e8109ec0 0x7fa6e81a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6eec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810fe50 con 0x7fa6e8074960 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6eec00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810fcd0 con 0x7fa6e8075f40 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6eec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810ffd0 con 0x7fa6e8109ec0 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8074960 0x7fa6e819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8074960 0x7fa6e819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49802/0 (socket says 172.21.15.46:49802) 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6e7e00640 1 -- 172.21.15.46:0/2671930977 learned_addr learned my addr 172.21.15.46:0/2671930977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6e7400640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8075f40 0x7fa6e819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.821+0000 7fa6ecc00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6e8109ec0 0x7fa6e81a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.822+0000 7fa6e7e00640 1 -- 172.21.15.46:0/2671930977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6e8109ec0 msgr2=0x7fa6e81a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.822+0000 7fa6e7e00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6e8109ec0 0x7fa6e81a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.822+0000 7fa6e7e00640 1 -- 172.21.15.46:0/2671930977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8075f40 msgr2=0x7fa6e819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.822+0000 7fa6e7e00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8075f40 0x7fa6e819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.822+0000 7fa6e7e00640 1 -- 172.21.15.46:0/2671930977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e81a2020 con 0x7fa6e8074960 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.822+0000 7fa6e7e00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8074960 0x7fa6e819a400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e9f0 tx=0x7fa6d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.822+0000 7fa6e4c00640 1 -- 172.21.15.46:0/2671930977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800cd80 con 0x7fa6e8074960 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.823+0000 7fa6e4c00640 1 -- 172.21.15.46:0/2671930977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d8010910 con 0x7fa6e8074960 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.823+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e81a2310 con 0x7fa6e8074960 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.823+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e81a2860 con 0x7fa6e8074960 2024-04-17T07:57:47.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.823+0000 7fa6e4c00640 1 -- 172.21.15.46:0/2671930977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8018930 con 0x7fa6e8074960 2024-04-17T07:57:47.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.824+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e8108c80 con 0x7fa6e8074960 2024-04-17T07:57:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.826+0000 7fa6e4c00640 1 -- 172.21.15.46:0/2671930977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa6d8010ac0 con 0x7fa6e8074960 2024-04-17T07:57:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.827+0000 7fa6e4c00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6cc0a3bb0 0x7fa6cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.828+0000 7fa6e4c00640 1 -- 172.21.15.46:0/2671930977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa6d80d0a20 con 0x7fa6e8074960 2024-04-17T07:57:47.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.828+0000 7fa6e7400640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6cc0a3bb0 0x7fa6cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:47.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.828+0000 7fa6e7400640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6cc0a3bb0 0x7fa6cc0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6e819b910 tx=0x7fa6d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:47.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:47.832+0000 7fa6e4c00640 1 -- 172.21.15.46:0/2671930977 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa6d809c9c0 con 0x7fa6e8074960 2024-04-17T07:57:48.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.039+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6e8062b60 con 0x7fa6e8074960 2024-04-17T07:57:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.043+0000 7fa6e4c00640 1 -- 172.21.15.46:0/2671930977 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7fa6d80a0cc0 con 0x7fa6e8074960 2024-04-17T07:57:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:48.043 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:57:48.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6cc0a3bb0 msgr2=0x7fa6cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6cc0a3bb0 0x7fa6cc0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6e819b910 tx=0x7fa6d000b410 comp rx=0 tx=0).stop 2024-04-17T07:57:48.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8074960 msgr2=0x7fa6e819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8074960 0x7fa6e819a400 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e9f0 tx=0x7fa6d800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 shutdown_connections 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa6cc0a3bb0 0x7fa6cc0a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6e8109ec0 0x7fa6e81a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8075f40 0x7fa6e819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 --2- 172.21.15.46:0/2671930977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e8074960 0x7fa6e819a400 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 >> 172.21.15.46:0/2671930977 conn(0x7fa6e80fd580 msgr2=0x7fa6e8105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 shutdown_connections 2024-04-17T07:57:48.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.046+0000 7fa6eec00640 1 -- 172.21.15.46:0/2671930977 wait complete. 2024-04-17T07:57:48.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:57:48.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:57:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 -- 172.21.15.46:0/854707772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341053a0 msgr2=0x7faf34105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 --2- 172.21.15.46:0/854707772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341053a0 0x7faf34105820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf20009860 tx=0x7faf2002bb70 comp rx=0 tx=0).stop 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 -- 172.21.15.46:0/854707772 shutdown_connections 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 --2- 172.21.15.46:0/854707772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf34105d60 0x7faf3410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 --2- 172.21.15.46:0/854707772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341053a0 0x7faf34105820 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 --2- 172.21.15.46:0/854707772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf341041b0 0x7faf341045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 -- 172.21.15.46:0/854707772 >> 172.21.15.46:0/854707772 conn(0x7faf340ff720 msgr2=0x7faf34101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 -- 172.21.15.46:0/854707772 shutdown_connections 2024-04-17T07:57:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.302+0000 7faf3a200640 1 -- 172.21.15.46:0/854707772 wait complete. 2024-04-17T07:57:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.303+0000 7faf3a200640 1 Processor -- start 2024-04-17T07:57:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf3a200640 1 -- start start 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf3a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341041b0 0x7faf3419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf3a200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf341053a0 0x7faf3419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf3a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34105d60 0x7faf341a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf3a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410fe40 con 0x7faf34105d60 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf3a200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410fcc0 con 0x7faf341053a0 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf3a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410ffc0 con 0x7faf341041b0 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf32a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf341053a0 0x7faf3419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf32a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf341053a0 0x7faf3419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51336/0 (socket says 172.21.15.46:51336) 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf32a00640 1 -- 172.21.15.46:0/1479731504 learned_addr learned my addr 172.21.15.46:0/1479731504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf33e00640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34105d60 0x7faf341a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.304+0000 7faf33400640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341041b0 0x7faf3419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf32a00640 1 -- 172.21.15.46:0/1479731504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341041b0 msgr2=0x7faf3419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf32a00640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341041b0 0x7faf3419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf32a00640 1 -- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34105d60 msgr2=0x7faf341a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf32a00640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34105d60 0x7faf341a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf32a00640 1 -- 172.21.15.46:0/1479731504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf341a20c0 con 0x7faf341053a0 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf32a00640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf341053a0 0x7faf3419a9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf200349e0 tx=0x7faf20032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf2be00640 1 -- 172.21.15.46:0/1479731504 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20041030 con 0x7faf341053a0 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf2be00640 1 -- 172.21.15.46:0/1479731504 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf2002c510 con 0x7faf341053a0 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf341a2350 con 0x7faf341053a0 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf2be00640 1 -- 172.21.15.46:0/1479731504 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20039040 con 0x7faf341053a0 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.305+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a2840 con 0x7faf341053a0 2024-04-17T07:57:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.307+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3410b550 con 0x7faf341053a0 2024-04-17T07:57:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.308+0000 7faf2be00640 1 -- 172.21.15.46:0/1479731504 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faf20044070 con 0x7faf341053a0 2024-04-17T07:57:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.310+0000 7faf2be00640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf180a3bb0 0x7faf180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.310+0000 7faf2be00640 1 -- 172.21.15.46:0/1479731504 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7faf200f1cd0 con 0x7faf341053a0 2024-04-17T07:57:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.310+0000 7faf33400640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf180a3bb0 0x7faf180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.311+0000 7faf33400640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf180a3bb0 0x7faf180a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faf34105200 tx=0x7faf2c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.314+0000 7faf2be00640 1 -- 172.21.15.46:0/1479731504 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faf200be2c0 con 0x7faf341053a0 2024-04-17T07:57:48.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:48 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2671930977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:48.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:48 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2671930977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:48.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.523+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf34062b60 con 0x7faf341053a0 2024-04-17T07:57:48.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.524+0000 7faf2be00640 1 -- 172.21.15.46:0/1479731504 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7faf200bdbc0 con 0x7faf341053a0 2024-04-17T07:57:48.523 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:57:48.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.528+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf180a3bb0 msgr2=0x7faf180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.528+0000 7faf3a200640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf180a3bb0 0x7faf180a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faf34105200 tx=0x7faf2c010040 comp rx=0 tx=0).stop 2024-04-17T07:57:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.528+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf341053a0 msgr2=0x7faf3419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.528+0000 7faf3a200640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf341053a0 0x7faf3419a9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf200349e0 tx=0x7faf20032450 comp rx=0 tx=0).stop 2024-04-17T07:57:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 shutdown_connections 2024-04-17T07:57:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faf180a3bb0 0x7faf180a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf34105d60 0x7faf341a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf341053a0 0x7faf3419a9e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 --2- 172.21.15.46:0/1479731504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf341041b0 0x7faf3419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 >> 172.21.15.46:0/1479731504 conn(0x7faf340ff720 msgr2=0x7faf34108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 shutdown_connections 2024-04-17T07:57:48.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.529+0000 7faf3a200640 1 -- 172.21.15.46:0/1479731504 wait complete. 2024-04-17T07:57:48.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:57:48.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:48 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2671930977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:48.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 -- 172.21.15.46:0/742942062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 msgr2=0x7f7f44109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 --2- 172.21.15.46:0/742942062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 0x7f7f44109a50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009900 tx=0x7f7f2c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:57:48.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 -- 172.21.15.46:0/742942062 shutdown_connections 2024-04-17T07:57:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 --2- 172.21.15.46:0/742942062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f44109f90 0x7f7f44110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 --2- 172.21.15.46:0/742942062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 0x7f7f44109a50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 --2- 172.21.15.46:0/742942062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f441083e0 0x7f7f44108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 -- 172.21.15.46:0/742942062 >> 172.21.15.46:0/742942062 conn(0x7f7f440773a0 msgr2=0x7f7f440797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 -- 172.21.15.46:0/742942062 shutdown_connections 2024-04-17T07:57:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.769+0000 7f7f43200640 1 -- 172.21.15.46:0/742942062 wait complete. 2024-04-17T07:57:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.770+0000 7f7f43200640 1 Processor -- start 2024-04-17T07:57:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.770+0000 7f7f43200640 1 -- start start 2024-04-17T07:57:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.770+0000 7f7f43200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f441083e0 0x7f7f4419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.770+0000 7f7f43200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 0x7f7f4419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f43200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f44109f90 0x7f7f441a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f43200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441140c0 con 0x7f7f44109f90 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f43200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44113f40 con 0x7f7f441095d0 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f43200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44114240 con 0x7f7f441083e0 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f441083e0 0x7f7f4419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f42800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f44109f90 0x7f7f441a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f441083e0 0x7f7f4419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:60926/0 (socket says 172.21.15.46:60926) 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 -- 172.21.15.46:0/3427642512 learned_addr learned my addr 172.21.15.46:0/3427642512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f3be00640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 0x7f7f4419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 -- 172.21.15.46:0/3427642512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 msgr2=0x7f7f4419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 0x7f7f4419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 -- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f44109f90 msgr2=0x7f7f441a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f44109f90 0x7f7f441a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f41e00640 1 -- 172.21.15.46:0/3427642512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f441a62f0 con 0x7f7f441083e0 2024-04-17T07:57:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.771+0000 7f7f42800640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f44109f90 0x7f7f441a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.772+0000 7f7f41e00640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f441083e0 0x7f7f4419e720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400e9c0 tx=0x7f7f3400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.772+0000 7f7f3aa00640 1 -- 172.21.15.46:0/3427642512 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400cd80 con 0x7f7f441083e0 2024-04-17T07:57:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.772+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f441a65e0 con 0x7f7f441083e0 2024-04-17T07:57:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.772+0000 7f7f3aa00640 1 -- 172.21.15.46:0/3427642512 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f340035f0 con 0x7f7f441083e0 2024-04-17T07:57:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.772+0000 7f7f3aa00640 1 -- 172.21.15.46:0/3427642512 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34010690 con 0x7f7f441083e0 2024-04-17T07:57:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.772+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f441a6b50 con 0x7f7f441083e0 2024-04-17T07:57:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.773+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4410f700 con 0x7f7f441083e0 2024-04-17T07:57:48.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.775+0000 7f7f3aa00640 1 -- 172.21.15.46:0/3427642512 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7f340037a0 con 0x7f7f441083e0 2024-04-17T07:57:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.777+0000 7f7f3aa00640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f280a3c30 0x7f7f280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.777+0000 7f7f3aa00640 1 -- 172.21.15.46:0/3427642512 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7f340d0d30 con 0x7f7f441083e0 2024-04-17T07:57:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.777+0000 7f7f3be00640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f280a3c30 0x7f7f280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.778+0000 7f7f3be00640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f280a3c30 0x7f7f280a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c02c260 tx=0x7f7f2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.780+0000 7f7f3aa00640 1 -- 172.21.15.46:0/3427642512 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7f3409d320 con 0x7f7f441083e0 2024-04-17T07:57:48.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.989+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f44062b60 con 0x7f7f441083e0 2024-04-17T07:57:48.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.990+0000 7f7f3aa00640 1 -- 172.21.15.46:0/3427642512 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f7f3409cc20 con 0x7f7f441083e0 2024-04-17T07:57:48.989 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:57:48.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.992+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f280a3c30 msgr2=0x7f7f280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.992+0000 7f7f43200640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f280a3c30 0x7f7f280a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c02c260 tx=0x7f7f2c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.992+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f441083e0 msgr2=0x7f7f4419e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:48.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.992+0000 7f7f43200640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f441083e0 0x7f7f4419e720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400e9c0 tx=0x7f7f3400ee80 comp rx=0 tx=0).stop 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 shutdown_connections 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f280a3c30 0x7f7f280a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f44109f90 0x7f7f441a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f441095d0 0x7f7f4419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 --2- 172.21.15.46:0/3427642512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f441083e0 0x7f7f4419e720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 >> 172.21.15.46:0/3427642512 conn(0x7f7f440773a0 msgr2=0x7f7f4410c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 shutdown_connections 2024-04-17T07:57:48.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:48.993+0000 7f7f43200640 1 -- 172.21.15.46:0/3427642512 wait complete. 2024-04-17T07:57:49.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:57:49.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.237+0000 7f1f0ae00640 1 -- 172.21.15.46:0/132492267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 msgr2=0x7f1f04103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.237+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/132492267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 0x7f1f04103440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002bc30 comp rx=0 tx=0).stop 2024-04-17T07:57:49.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.238+0000 7f1f0ae00640 1 -- 172.21.15.46:0/132492267 shutdown_connections 2024-04-17T07:57:49.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.238+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/132492267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04103980 0x7f1f0410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.238+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/132492267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 0x7f1f04103440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.238+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/132492267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 0x7f1f041021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.238+0000 7f1f0ae00640 1 -- 172.21.15.46:0/132492267 >> 172.21.15.46:0/132492267 conn(0x7f1f040fd340 msgr2=0x7f1f040ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:49.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.238+0000 7f1f0ae00640 1 -- 172.21.15.46:0/132492267 shutdown_connections 2024-04-17T07:57:49.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.238+0000 7f1f0ae00640 1 -- 172.21.15.46:0/132492267 wait complete. 2024-04-17T07:57:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 Processor -- start 2024-04-17T07:57:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 -- start start 2024-04-17T07:57:49.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 0x7f1f04075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 0x7f1f04074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04103980 0x7f1f04074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410daa0 con 0x7f1f04103980 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410d920 con 0x7f1f04102fc0 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.239+0000 7f1f0ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410dc20 con 0x7f1f04101dd0 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 0x7f1f04075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 0x7f1f04074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 0x7f1f04075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:60952/0 (socket says 172.21.15.46:60952) 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 -- 172.21.15.46:0/728598361 learned_addr learned my addr 172.21.15.46:0/728598361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f0a400640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04103980 0x7f1f04074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 -- 172.21.15.46:0/728598361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 msgr2=0x7f1f04074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 0x7f1f04074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 -- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04103980 msgr2=0x7f1f04074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04103980 0x7f1f04074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.240+0000 7f1f09a00640 1 -- 172.21.15.46:0/728598361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f04075120 con 0x7f1f04101dd0 2024-04-17T07:57:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.241+0000 7f1f09a00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 0x7f1f04075d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800e9c0 tx=0x7f1ef800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:49.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.241+0000 7f1f02000640 1 -- 172.21.15.46:0/728598361 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800cd80 con 0x7f1f04101dd0 2024-04-17T07:57:49.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.241+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f041a21d0 con 0x7f1f04101dd0 2024-04-17T07:57:49.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.241+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f041a26f0 con 0x7f1f04101dd0 2024-04-17T07:57:49.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.241+0000 7f1f02000640 1 -- 172.21.15.46:0/728598361 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef80035f0 con 0x7f1f04101dd0 2024-04-17T07:57:49.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.241+0000 7f1f02000640 1 -- 172.21.15.46:0/728598361 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8010670 con 0x7f1f04101dd0 2024-04-17T07:57:49.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.242+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0404fcf0 con 0x7f1f04101dd0 2024-04-17T07:57:49.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.244+0000 7f1f02000640 1 -- 172.21.15.46:0/728598361 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1ef80037a0 con 0x7f1f04101dd0 2024-04-17T07:57:49.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.245+0000 7f1f02000640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1eec0a3ba0 0x7f1eec0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.246+0000 7f1f02000640 1 -- 172.21.15.46:0/728598361 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f1ef80d0be0 con 0x7f1f04101dd0 2024-04-17T07:57:49.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.246+0000 7f1f09000640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1eec0a3ba0 0x7f1eec0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.247+0000 7f1f09000640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1eec0a3ba0 0x7f1eec0a6060 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1ef002c240 tx=0x7f1ef002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:49.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.250+0000 7f1f02000640 1 -- 172.21.15.46:0/728598361 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1ef809d1d0 con 0x7f1f04101dd0 2024-04-17T07:57:49.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:49 smithi046 ceph-mon[32038]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:49.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:49 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1479731504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:57:49.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:49 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3427642512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:57:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.461+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f04062b60 con 0x7f1f04101dd0 2024-04-17T07:57:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.462+0000 7f1f02000640 1 -- 172.21.15.46:0/728598361 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f1ef809cad0 con 0x7f1f04101dd0 2024-04-17T07:57:49.461 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:57:49.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1eec0a3ba0 msgr2=0x7f1eec0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1eec0a3ba0 0x7f1eec0a6060 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1ef002c240 tx=0x7f1ef002df90 comp rx=0 tx=0).stop 2024-04-17T07:57:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 msgr2=0x7f1f04075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 0x7f1f04075d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800e9c0 tx=0x7f1ef800ee80 comp rx=0 tx=0).stop 2024-04-17T07:57:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 shutdown_connections 2024-04-17T07:57:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1eec0a3ba0 0x7f1eec0a6060 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04103980 0x7f1f04074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f04102fc0 0x7f1f04074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 --2- 172.21.15.46:0/728598361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f04101dd0 0x7f1f04075d10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 >> 172.21.15.46:0/728598361 conn(0x7f1f040fd340 msgr2=0x7f1f04106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 shutdown_connections 2024-04-17T07:57:49.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.465+0000 7f1f0ae00640 1 -- 172.21.15.46:0/728598361 wait complete. 2024-04-17T07:57:49.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:49 smithi053 ceph-mon[36437]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:49.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:49 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1479731504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:57:49.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:49 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3427642512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:57:49.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:57:49.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:49 smithi062 ceph-mon[36507]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:49.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:49 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1479731504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:57:49.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:49 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3427642512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.718+0000 7fa1fda00640 1 -- 172.21.15.46:0/2612906373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8076920 msgr2=0x7fa1f8076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.718+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2612906373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8076920 0x7fa1f8076d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009900 tx=0x7fa1e802bc40 comp rx=0 tx=0).stop 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.718+0000 7fa1fda00640 1 -- 172.21.15.46:0/2612906373 shutdown_connections 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.718+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2612906373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8078500 0x7fa1f8078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.718+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2612906373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f8077b60 0x7fa1f8077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.718+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2612906373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8076920 0x7fa1f8076d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.718+0000 7fa1fda00640 1 -- 172.21.15.46:0/2612906373 >> 172.21.15.46:0/2612906373 conn(0x7fa1f80ff570 msgr2=0x7fa1f8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:49.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.719+0000 7fa1fda00640 1 -- 172.21.15.46:0/2612906373 shutdown_connections 2024-04-17T07:57:49.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.719+0000 7fa1fda00640 1 -- 172.21.15.46:0/2612906373 wait complete. 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.720+0000 7fa1fda00640 1 Processor -- start 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.720+0000 7fa1fda00640 1 -- start start 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.720+0000 7fa1fda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8076920 0x7fa1f819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.720+0000 7fa1fda00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8077b60 0x7fa1f819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.720+0000 7fa1fda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f8078500 0x7fa1f81a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1fda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f81140b0 con 0x7fa1f8076920 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1fda00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8113f30 con 0x7fa1f8077b60 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1fda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8114230 con 0x7fa1f8078500 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8076920 0x7fa1f819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8076920 0x7fa1f819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38268/0 (socket says 172.21.15.46:38268) 2024-04-17T07:57:49.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 -- 172.21.15.46:0/2797073269 learned_addr learned my addr 172.21.15.46:0/2797073269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:49.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1fd000640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f8078500 0x7fa1f81a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1efe00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8077b60 0x7fa1f819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 -- 172.21.15.46:0/2797073269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f8078500 msgr2=0x7fa1f81a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f8078500 0x7fa1f81a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 -- 172.21.15.46:0/2797073269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8077b60 msgr2=0x7fa1f819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8077b60 0x7fa1f819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.721+0000 7fa1f7e00640 1 -- 172.21.15.46:0/2797073269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f81a6360 con 0x7fa1f8076920 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.722+0000 7fa1f7e00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8076920 0x7fa1f819e740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802c8f0 tx=0x7fa1e802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.722+0000 7fa1f5600640 1 -- 172.21.15.46:0/2797073269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8041070 con 0x7fa1f8076920 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.722+0000 7fa1f5600640 1 -- 172.21.15.46:0/2797073269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8004450 con 0x7fa1f8076920 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.722+0000 7fa1f5600640 1 -- 172.21.15.46:0/2797073269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003df0 con 0x7fa1f8076920 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.722+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f81a65f0 con 0x7fa1f8076920 2024-04-17T07:57:49.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.723+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f81a6a90 con 0x7fa1f8076920 2024-04-17T07:57:49.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.724+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f8110950 con 0x7fa1f8076920 2024-04-17T07:57:49.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.726+0000 7fa1f5600640 1 -- 172.21.15.46:0/2797073269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa1e8044070 con 0x7fa1f8076920 2024-04-17T07:57:49.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.728+0000 7fa1f5600640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa1dc0a3c30 0x7fa1dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:49.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.728+0000 7fa1f5600640 1 -- 172.21.15.46:0/2797073269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa1e80f2280 con 0x7fa1f8076920 2024-04-17T07:57:49.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.728+0000 7fa1efe00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa1dc0a3c30 0x7fa1dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:49.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.729+0000 7fa1efe00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa1dc0a3c30 0x7fa1dc0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa1f819fc50 tx=0x7fa1e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.732+0000 7fa1f5600640 1 -- 172.21.15.46:0/2797073269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa1e80be870 con 0x7fa1f8076920 2024-04-17T07:57:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.940+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa1f8062b90 con 0x7fa1f8076920 2024-04-17T07:57:49.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.941+0000 7fa1f5600640 1 -- 172.21.15.46:0/2797073269 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fa1e80be170 con 0x7fa1f8076920 2024-04-17T07:57:49.940 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa1dc0a3c30 msgr2=0x7fa1dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa1dc0a3c30 0x7fa1dc0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa1f819fc50 tx=0x7fa1e0005f90 comp rx=0 tx=0).stop 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8076920 msgr2=0x7fa1f819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8076920 0x7fa1f819e740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802c8f0 tx=0x7fa1e802c660 comp rx=0 tx=0).stop 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 shutdown_connections 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa1dc0a3c30 0x7fa1dc0a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f8078500 0x7fa1f81a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8077b60 0x7fa1f819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 --2- 172.21.15.46:0/2797073269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1f8076920 0x7fa1f819e740 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 >> 172.21.15.46:0/2797073269 conn(0x7fa1f80ff570 msgr2=0x7fa1f8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 shutdown_connections 2024-04-17T07:57:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:49.944+0000 7fa1fda00640 1 -- 172.21.15.46:0/2797073269 wait complete. 2024-04-17T07:57:49.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:57:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 -- 172.21.15.46:0/3881183228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8884108500 msgr2=0x7f8884108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 --2- 172.21.15.46:0/3881183228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8884108500 0x7f8884108920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8874009900 tx=0x7f887402bc50 comp rx=0 tx=0).stop 2024-04-17T07:57:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 -- 172.21.15.46:0/3881183228 shutdown_connections 2024-04-17T07:57:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 --2- 172.21.15.46:0/3881183228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888410a060 0x7f88841108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 --2- 172.21.15.46:0/3881183228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88841096a0 0x7f8884109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 --2- 172.21.15.46:0/3881183228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8884108500 0x7f8884108920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 -- 172.21.15.46:0/3881183228 >> 172.21.15.46:0/3881183228 conn(0x7f8884077450 msgr2=0x7f8884079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 -- 172.21.15.46:0/3881183228 shutdown_connections 2024-04-17T07:57:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.178+0000 7f8883e00640 1 -- 172.21.15.46:0/3881183228 wait complete. 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.179+0000 7f8883e00640 1 Processor -- start 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.179+0000 7f8883e00640 1 -- start start 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8883e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8884108500 0x7f888419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8883e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88841096a0 0x7f888419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8883e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f888410a060 0x7f88841a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8883e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8884114160 con 0x7f888410a060 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8883e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8884113fe0 con 0x7f88841096a0 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8883e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88841142e0 con 0x7f8884108500 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8882a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8884108500 0x7f888419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8882a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8884108500 0x7f888419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:60984/0 (socket says 172.21.15.46:60984) 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8882a00640 1 -- 172.21.15.46:0/384916181 learned_addr learned my addr 172.21.15.46:0/384916181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8883400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f888410a060 0x7f88841a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.180+0000 7f8882000640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88841096a0 0x7f888419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8882a00640 1 -- 172.21.15.46:0/384916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88841096a0 msgr2=0x7f888419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8882a00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88841096a0 0x7f888419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8882a00640 1 -- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f888410a060 msgr2=0x7f88841a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8882a00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f888410a060 0x7f88841a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8882a00640 1 -- 172.21.15.46:0/384916181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88841a63f0 con 0x7f8884108500 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8883400640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f888410a060 0x7f88841a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8882000640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88841096a0 0x7f888419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8882a00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8884108500 0x7f888419e7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f887402c260 tx=0x7f8874005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f887b400640 1 -- 172.21.15.46:0/384916181 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874041070 con 0x7f8884108500 2024-04-17T07:57:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88841a6680 con 0x7f8884108500 2024-04-17T07:57:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f887b400640 1 -- 172.21.15.46:0/384916181 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88740054e0 con 0x7f8884108500 2024-04-17T07:57:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f887b400640 1 -- 172.21.15.46:0/384916181 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88740346a0 con 0x7f8884108500 2024-04-17T07:57:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.181+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88841a6c20 con 0x7f8884108500 2024-04-17T07:57:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.183+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888410f7d0 con 0x7f8884108500 2024-04-17T07:57:50.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.184+0000 7f887b400640 1 -- 172.21.15.46:0/384916181 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8874044070 con 0x7f8884108500 2024-04-17T07:57:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.186+0000 7f887b400640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88680a3c20 0x7f88680a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.186+0000 7f887b400640 1 -- 172.21.15.46:0/384916181 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f88740f2320 con 0x7f8884108500 2024-04-17T07:57:50.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.186+0000 7f8882000640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88680a3c20 0x7f88680a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.187+0000 7f8882000640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88680a3c20 0x7f88680a60e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f888419fce0 tx=0x7f886c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.190+0000 7f887b400640 1 -- 172.21.15.46:0/384916181 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8874003e20 con 0x7f8884108500 2024-04-17T07:57:50.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:50 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/728598361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:57:50.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:50 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2797073269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:57:50.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.449+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f888402d5d0 con 0x7f8884108500 2024-04-17T07:57:50.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.450+0000 7f887b400640 1 -- 172.21.15.46:0/384916181 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f88740be2c0 con 0x7f8884108500 2024-04-17T07:57:50.450 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:50.450 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:54.519826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24556,24542]},"id":1}]} 2024-04-17T07:57:50.450 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88680a3c20 msgr2=0x7f88680a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88680a3c20 0x7f88680a60e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f888419fce0 tx=0x7f886c00b410 comp rx=0 tx=0).stop 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8884108500 msgr2=0x7f888419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8884108500 0x7f888419e7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f887402c260 tx=0x7f8874005dd0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 shutdown_connections 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f88680a3c20 0x7f88680a60e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f888410a060 0x7f88841a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88841096a0 0x7f888419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 --2- 172.21.15.46:0/384916181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8884108500 0x7f888419e7d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.453+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 >> 172.21.15.46:0/384916181 conn(0x7f8884077450 msgr2=0x7f888410c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.454+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 shutdown_connections 2024-04-17T07:57:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.454+0000 7f8883e00640 1 -- 172.21.15.46:0/384916181 wait complete. 2024-04-17T07:57:50.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:50 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/728598361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:57:50.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:50 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2797073269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:57:50.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:57:50.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:50 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/728598361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:57:50.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:50 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2797073269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 -- 172.21.15.46:0/722842712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 msgr2=0x7f30a810a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 --2- 172.21.15.46:0/722842712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 0x7f30a810a380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009cd0 tx=0x7f30a002c0e0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 -- 172.21.15.46:0/722842712 shutdown_connections 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 --2- 172.21.15.46:0/722842712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 0x7f30a810a380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 --2- 172.21.15.46:0/722842712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8103130 0x7f30a81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 --2- 172.21.15.46:0/722842712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a8100160 0x7f30a8100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 -- 172.21.15.46:0/722842712 >> 172.21.15.46:0/722842712 conn(0x7f30a80fbad0 msgr2=0x7f30a80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.701+0000 7f30ace00640 1 -- 172.21.15.46:0/722842712 shutdown_connections 2024-04-17T07:57:50.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.702+0000 7f30ace00640 1 -- 172.21.15.46:0/722842712 wait complete. 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.702+0000 7f30ace00640 1 Processor -- start 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30ace00640 1 -- start start 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30ace00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8100160 0x7f30a8102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30ace00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a8103130 0x7f30a81010e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30ace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 0x7f30a8101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30ace00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30a810db10 con 0x7f30a8103af0 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30ace00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f30a810d990 con 0x7f30a8100160 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30ace00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30a810dc90 con 0x7f30a8103130 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 0x7f30a8101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30a6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a8103130 0x7f30a81010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30a7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8100160 0x7f30a8102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30a7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8100160 0x7f30a8102aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51424/0 (socket says 172.21.15.46:51424) 2024-04-17T07:57:50.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.703+0000 7f30a7400640 1 -- 172.21.15.46:0/1543218481 learned_addr learned my addr 172.21.15.46:0/1543218481 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30a7400640 1 -- 172.21.15.46:0/1543218481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a8103130 msgr2=0x7f30a81010e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30a7400640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a8103130 0x7f30a81010e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30a7400640 1 -- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 msgr2=0x7f30a8101620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30a7400640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 0x7f30a8101620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30a7400640 1 -- 172.21.15.46:0/1543218481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a8101eb0 con 0x7f30a8100160 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30a7e00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 0x7f30a8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30a7400640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8100160 0x7f30a8102aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f309800e9c0 tx=0x7f309800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f309fe00640 1 -- 172.21.15.46:0/1543218481 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309800cd80 con 0x7f30a8100160 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f309fe00640 1 -- 172.21.15.46:0/1543218481 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980035f0 con 0x7f30a8100160 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.704+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a81a2270 con 0x7f30a8100160 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.705+0000 7f309fe00640 1 -- 172.21.15.46:0/1543218481 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098010690 con 0x7f30a8100160 2024-04-17T07:57:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.705+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a81a27c0 con 0x7f30a8100160 2024-04-17T07:57:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.706+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a8109260 con 0x7f30a8100160 2024-04-17T07:57:50.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.708+0000 7f309fe00640 1 -- 172.21.15.46:0/1543218481 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f30980037a0 con 0x7f30a8100160 2024-04-17T07:57:50.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.709+0000 7f309fe00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f308c0a3ba0 0x7f308c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:50.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.709+0000 7f309fe00640 1 -- 172.21.15.46:0/1543218481 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f30980d0c50 con 0x7f30a8100160 2024-04-17T07:57:50.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.709+0000 7f30a6a00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f308c0a3ba0 0x7f308c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.710+0000 7f30a6a00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f308c0a3ba0 0x7f308c0a6060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30940095f0 tx=0x7f3094005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:50.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.714+0000 7f309fe00640 1 -- 172.21.15.46:0/1543218481 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f309809d240 con 0x7f30a8100160 2024-04-17T07:57:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.923+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30a8062b60 con 0x7f30a8100160 2024-04-17T07:57:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.926+0000 7f309fe00640 1 -- 172.21.15.46:0/1543218481 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7f309809cb40 con 0x7f30a8100160 2024-04-17T07:57:50.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:50.926 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:57:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.928+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f308c0a3ba0 msgr2=0x7f308c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.928+0000 7f30ace00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f308c0a3ba0 0x7f308c0a6060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30940095f0 tx=0x7f3094005f90 comp rx=0 tx=0).stop 2024-04-17T07:57:50.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.928+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8100160 msgr2=0x7f30a8102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.928+0000 7f30ace00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8100160 0x7f30a8102aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f309800e9c0 tx=0x7f309800ee80 comp rx=0 tx=0).stop 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 shutdown_connections 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f308c0a3ba0 0x7f308c0a6060 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a8103af0 0x7f30a8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a8103130 0x7f30a81010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 --2- 172.21.15.46:0/1543218481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8100160 0x7f30a8102aa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 >> 172.21.15.46:0/1543218481 conn(0x7f30a80fbad0 msgr2=0x7f30a8106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 shutdown_connections 2024-04-17T07:57:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:50.929+0000 7f30ace00640 1 -- 172.21.15.46:0/1543218481 wait complete. 2024-04-17T07:57:50.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:57:50.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:57:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 -- 172.21.15.46:0/1672152259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 msgr2=0x7f859c10c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 --2- 172.21.15.46:0/1672152259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 0x7f859c10c400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8594009900 tx=0x7f859402bc30 comp rx=0 tx=0).stop 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 -- 172.21.15.46:0/1672152259 shutdown_connections 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 --2- 172.21.15.46:0/1672152259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 0x7f859c10c400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 --2- 172.21.15.46:0/1672152259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859c1051b0 0x7f859c105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 --2- 172.21.15.46:0/1672152259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 0x7f859c1043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 -- 172.21.15.46:0/1672152259 >> 172.21.15.46:0/1672152259 conn(0x7f859c0ff570 msgr2=0x7f859c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 -- 172.21.15.46:0/1672152259 shutdown_connections 2024-04-17T07:57:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.182+0000 7f85a1000640 1 -- 172.21.15.46:0/1672152259 wait complete. 2024-04-17T07:57:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.183+0000 7f85a1000640 1 Processor -- start 2024-04-17T07:57:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f85a1000640 1 -- start start 2024-04-17T07:57:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f85a1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 0x7f859c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f85a1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859c1051b0 0x7f859c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f85a1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 0x7f859c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f85a1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10fca0 con 0x7f859c1051b0 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f85a1000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10fb20 con 0x7f859c105b70 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f85a1000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10fe20 con 0x7f859c103fc0 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f859b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 0x7f859c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f859b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 0x7f859c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:32798/0 (socket says 172.21.15.46:32798) 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f859b400640 1 -- 172.21.15.46:0/282631171 learned_addr learned my addr 172.21.15.46:0/282631171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f8592a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859c1051b0 0x7f859c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.184+0000 7f859be00640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 0x7f859c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f859b400640 1 -- 172.21.15.46:0/282631171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 msgr2=0x7f859c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f859b400640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 0x7f859c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f859b400640 1 -- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859c1051b0 msgr2=0x7f859c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f859b400640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859c1051b0 0x7f859c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f859b400640 1 -- 172.21.15.46:0/282631171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c1a1e80 con 0x7f859c103fc0 2024-04-17T07:57:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f859b400640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 0x7f859c19a260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f858c00eb70 tx=0x7f858c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f8598c00640 1 -- 172.21.15.46:0/282631171 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c003b70 con 0x7f859c103fc0 2024-04-17T07:57:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f8598c00640 1 -- 172.21.15.46:0/282631171 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f858c00ee70 con 0x7f859c103fc0 2024-04-17T07:57:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f8598c00640 1 -- 172.21.15.46:0/282631171 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c010830 con 0x7f859c103fc0 2024-04-17T07:57:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c1a2170 con 0x7f859c103fc0 2024-04-17T07:57:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.185+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c1a2670 con 0x7f859c103fc0 2024-04-17T07:57:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.187+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c10b280 con 0x7f859c103fc0 2024-04-17T07:57:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.188+0000 7f8598c00640 1 -- 172.21.15.46:0/282631171 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f858c004020 con 0x7f859c103fc0 2024-04-17T07:57:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.190+0000 7f8598c00640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f85800a3c20 0x7f85800a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.190+0000 7f8598c00640 1 -- 172.21.15.46:0/282631171 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f858c0d0df0 con 0x7f859c103fc0 2024-04-17T07:57:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.190+0000 7f8592a00640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f85800a3c20 0x7f85800a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.191+0000 7f8592a00640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f85800a3c20 0x7f85800a60e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f859c19b770 tx=0x7f8584005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.194+0000 7f8598c00640 1 -- 172.21.15.46:0/282631171 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f858c09d3e0 con 0x7f859c103fc0 2024-04-17T07:57:51.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:51 smithi046 ceph-mon[32038]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:51 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/384916181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:51.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:51 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1543218481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.406+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f859c062b90 con 0x7f859c103fc0 2024-04-17T07:57:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.407+0000 7f8598c00640 1 -- 172.21.15.46:0/282631171 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f858c09cce0 con 0x7f859c103fc0 2024-04-17T07:57:51.407 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f85800a3c20 msgr2=0x7f85800a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f85800a3c20 0x7f85800a60e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f859c19b770 tx=0x7f8584005f90 comp rx=0 tx=0).stop 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 msgr2=0x7f859c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 0x7f859c19a260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f858c00eb70 tx=0x7f858c00c6a0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 shutdown_connections 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f85800a3c20 0x7f85800a60e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f859c105b70 0x7f859c1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859c1051b0 0x7f859c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 --2- 172.21.15.46:0/282631171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f859c103fc0 0x7f859c19a260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 >> 172.21.15.46:0/282631171 conn(0x7f859c0ff570 msgr2=0x7f859c108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.410+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 shutdown_connections 2024-04-17T07:57:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.411+0000 7f85a1000640 1 -- 172.21.15.46:0/282631171 wait complete. 2024-04-17T07:57:51.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:57:51.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:51 smithi053 ceph-mon[36437]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:51.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:51 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/384916181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:51.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:51 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1543218481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:51.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:51 smithi062 ceph-mon[36507]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:51.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:51 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/384916181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:51.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:51 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1543218481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 -- 172.21.15.46:0/1338942351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341041c0 msgr2=0x7f78341045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 --2- 172.21.15.46:0/1338942351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341041c0 0x7f78341045e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7824009860 tx=0x7f782402bb70 comp rx=0 tx=0).stop 2024-04-17T07:57:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 -- 172.21.15.46:0/1338942351 shutdown_connections 2024-04-17T07:57:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 --2- 172.21.15.46:0/1338942351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7834105d70 0x7f783410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 --2- 172.21.15.46:0/1338942351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78341053b0 0x7f7834105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 --2- 172.21.15.46:0/1338942351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341041c0 0x7f78341045e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 -- 172.21.15.46:0/1338942351 >> 172.21.15.46:0/1338942351 conn(0x7f78340ff750 msgr2=0x7f7834101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 -- 172.21.15.46:0/1338942351 shutdown_connections 2024-04-17T07:57:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.668+0000 7f7838a00640 1 -- 172.21.15.46:0/1338942351 wait complete. 2024-04-17T07:57:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.669+0000 7f7838a00640 1 Processor -- start 2024-04-17T07:57:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7838a00640 1 -- start start 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7838a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78341041c0 0x7f783419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7838a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341053b0 0x7f783419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7838a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7834105d70 0x7f78341a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7838a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f783410fe20 con 0x7f7834105d70 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7838a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f783410fca0 con 0x7f78341053b0 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7838a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f783410ffa0 con 0x7f78341041c0 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7831400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341053b0 0x7f783419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7831400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341053b0 0x7f783419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51450/0 (socket says 172.21.15.46:51450) 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7832800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7834105d70 0x7f78341a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.670+0000 7f7831400640 1 -- 172.21.15.46:0/35905128 learned_addr learned my addr 172.21.15.46:0/35905128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7831e00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78341041c0 0x7f783419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7831400640 1 -- 172.21.15.46:0/35905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78341041c0 msgr2=0x7f783419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7831400640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78341041c0 0x7f783419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7831400640 1 -- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7834105d70 msgr2=0x7f78341a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7831400640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7834105d70 0x7f78341a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7831400640 1 -- 172.21.15.46:0/35905128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78341a2140 con 0x7f78341053b0 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7832800640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7834105d70 0x7f78341a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:57:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7831400640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341053b0 0x7f783419aa60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f782000e9c0 tx=0x7f782000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:51.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f782aa00640 1 -- 172.21.15.46:0/35905128 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782000cde0 con 0x7f78341053b0 2024-04-17T07:57:51.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f782aa00640 1 -- 172.21.15.46:0/35905128 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78200035f0 con 0x7f78341053b0 2024-04-17T07:57:51.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78341a2430 con 0x7f78341053b0 2024-04-17T07:57:51.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.671+0000 7f782aa00640 1 -- 172.21.15.46:0/35905128 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820010650 con 0x7f78341053b0 2024-04-17T07:57:51.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.672+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78341a29d0 con 0x7f78341053b0 2024-04-17T07:57:51.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.673+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783410b500 con 0x7f78341053b0 2024-04-17T07:57:51.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.676+0000 7f782aa00640 1 -- 172.21.15.46:0/35905128 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f78200037a0 con 0x7f78341053b0 2024-04-17T07:57:51.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.677+0000 7f782aa00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78180a3c30 0x7f78180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:51.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.677+0000 7f782aa00640 1 -- 172.21.15.46:0/35905128 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f78200d0d20 con 0x7f78341053b0 2024-04-17T07:57:51.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.677+0000 7f7831e00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78180a3c30 0x7f78180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:51.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.678+0000 7f7831e00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78180a3c30 0x7f78180a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78240349e0 tx=0x7f7824034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:51.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.679+0000 7f782aa00640 1 -- 172.21.15.46:0/35905128 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f782009d310 con 0x7f78341053b0 2024-04-17T07:57:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.886+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f783402d760 con 0x7f78341053b0 2024-04-17T07:57:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.887+0000 7f782aa00640 1 -- 172.21.15.46:0/35905128 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f782009cc10 con 0x7f78341053b0 2024-04-17T07:57:51.886 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:57:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78180a3c30 msgr2=0x7f78180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78180a3c30 0x7f78180a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78240349e0 tx=0x7f7824034ab0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341053b0 msgr2=0x7f783419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341053b0 0x7f783419aa60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f782000e9c0 tx=0x7f782000ee80 comp rx=0 tx=0).stop 2024-04-17T07:57:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 shutdown_connections 2024-04-17T07:57:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f78180a3c30 0x7f78180a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7834105d70 0x7f78341a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78341053b0 0x7f783419aa60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 --2- 172.21.15.46:0/35905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78341041c0 0x7f783419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 >> 172.21.15.46:0/35905128 conn(0x7f78340ff750 msgr2=0x7f78341086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 shutdown_connections 2024-04-17T07:57:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:51.890+0000 7f7838a00640 1 -- 172.21.15.46:0/35905128 wait complete. 2024-04-17T07:57:51.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.141+0000 7f5447e00640 1 -- 172.21.15.46:0/3593164035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 msgr2=0x7f5448104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.141+0000 7f5447e00640 1 --2- 172.21.15.46:0/3593164035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f5448104400 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5438009900 tx=0x7f543802bc40 comp rx=0 tx=0).stop 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.141+0000 7f5447e00640 1 -- 172.21.15.46:0/3593164035 shutdown_connections 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.141+0000 7f5447e00640 1 --2- 172.21.15.46:0/3593164035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5448105b90 0x7f544810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.141+0000 7f5447e00640 1 --2- 172.21.15.46:0/3593164035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54481051d0 0x7f5448105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.141+0000 7f5447e00640 1 --2- 172.21.15.46:0/3593164035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f5448104400 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.141+0000 7f5447e00640 1 -- 172.21.15.46:0/3593164035 >> 172.21.15.46:0/3593164035 conn(0x7f54480ff570 msgr2=0x7f5448101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:52.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.142+0000 7f5447e00640 1 -- 172.21.15.46:0/3593164035 shutdown_connections 2024-04-17T07:57:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.142+0000 7f5447e00640 1 -- 172.21.15.46:0/3593164035 wait complete. 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 Processor -- start 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 -- start start 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f544819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54481051d0 0x7f544819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5448105b90 0x7f54481a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f544810fcc0 con 0x7f54481051d0 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f544810fb40 con 0x7f5448103fe0 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5447e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f544810fe40 con 0x7f5448105b90 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5446000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54481051d0 0x7f544819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.143+0000 7f5446a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f544819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54481051d0 0x7f544819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38364/0 (socket says 172.21.15.46:38364) 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f544819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51464/0 (socket says 172.21.15.46:51464) 2024-04-17T07:57:52.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446000640 1 -- 172.21.15.46:0/2788543567 learned_addr learned my addr 172.21.15.46:0/2788543567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5447400640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5448105b90 0x7f54481a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446a00640 1 -- 172.21.15.46:0/2788543567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5448105b90 msgr2=0x7f54481a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446a00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5448105b90 0x7f54481a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446a00640 1 -- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54481051d0 msgr2=0x7f544819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446a00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54481051d0 0x7f544819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446a00640 1 -- 172.21.15.46:0/2788543567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54481a1ec0 con 0x7f5448103fe0 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446000640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54481051d0 0x7f544819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.144+0000 7f5446a00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f544819a2a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5438009230 tx=0x7f5438005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.145+0000 7f543f400640 1 -- 172.21.15.46:0/2788543567 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438041070 con 0x7f5448103fe0 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.145+0000 7f543f400640 1 -- 172.21.15.46:0/2788543567 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54380054e0 con 0x7f5448103fe0 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.145+0000 7f543f400640 1 -- 172.21.15.46:0/2788543567 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438034440 con 0x7f5448103fe0 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.145+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54481a2150 con 0x7f5448103fe0 2024-04-17T07:57:52.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.145+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54481a2610 con 0x7f5448103fe0 2024-04-17T07:57:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.146+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544810b300 con 0x7f5448103fe0 2024-04-17T07:57:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.148+0000 7f543f400640 1 -- 172.21.15.46:0/2788543567 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5438044070 con 0x7f5448103fe0 2024-04-17T07:57:52.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.149+0000 7f543f400640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f542c0a3ba0 0x7f542c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.149+0000 7f543f400640 1 -- 172.21.15.46:0/2788543567 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f54380f22f0 con 0x7f5448103fe0 2024-04-17T07:57:52.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.149+0000 7f5446000640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f542c0a3ba0 0x7f542c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.150+0000 7f5446000640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f542c0a3ba0 0x7f542c0a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f544819b7b0 tx=0x7f543400b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:52.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.153+0000 7f543f400640 1 -- 172.21.15.46:0/2788543567 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f54380be290 con 0x7f5448103fe0 2024-04-17T07:57:52.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:52 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/282631171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:57:52.361 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:52 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/35905128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:57:52.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.363+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5448062b60 con 0x7f5448103fe0 2024-04-17T07:57:52.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.364+0000 7f543f400640 1 -- 172.21.15.46:0/2788543567 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f54380c2590 con 0x7f5448103fe0 2024-04-17T07:57:52.363 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:57:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.366+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f542c0a3ba0 msgr2=0x7f542c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.366+0000 7f5447e00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f542c0a3ba0 0x7f542c0a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f544819b7b0 tx=0x7f543400b480 comp rx=0 tx=0).stop 2024-04-17T07:57:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 msgr2=0x7f544819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f544819a2a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5438009230 tx=0x7f5438005d50 comp rx=0 tx=0).stop 2024-04-17T07:57:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 shutdown_connections 2024-04-17T07:57:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f542c0a3ba0 0x7f542c0a6060 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5448105b90 0x7f54481a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54481051d0 0x7f544819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 --2- 172.21.15.46:0/2788543567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5448103fe0 0x7f544819a2a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 >> 172.21.15.46:0/2788543567 conn(0x7f54480ff570 msgr2=0x7f54481084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 shutdown_connections 2024-04-17T07:57:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.367+0000 7f5447e00640 1 -- 172.21.15.46:0/2788543567 wait complete. 2024-04-17T07:57:52.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:57:52.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:52 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/282631171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:57:52.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:52 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/35905128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:57:52.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:52 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/282631171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:57:52.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:52 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/35905128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:57:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 -- 172.21.15.46:0/1923005480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107aa0 msgr2=0x7fcd00109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 --2- 172.21.15.46:0/1923005480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107aa0 0x7fcd00109ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ca0 tx=0x7fccf802c100 comp rx=0 tx=0).stop 2024-04-17T07:57:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 -- 172.21.15.46:0/1923005480 shutdown_connections 2024-04-17T07:57:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 --2- 172.21.15.46:0/1923005480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107aa0 0x7fcd00109ef0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 --2- 172.21.15.46:0/1923005480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00105170 0x7fcd00107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 --2- 172.21.15.46:0/1923005480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd00102840 0x7fcd00104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 -- 172.21.15.46:0/1923005480 >> 172.21.15.46:0/1923005480 conn(0x7fcd000fc250 msgr2=0x7fcd000fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 -- 172.21.15.46:0/1923005480 shutdown_connections 2024-04-17T07:57:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.620+0000 7fcd07000640 1 -- 172.21.15.46:0/1923005480 wait complete. 2024-04-17T07:57:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 Processor -- start 2024-04-17T07:57:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 -- start start 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00102840 0x7fcd0019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00105170 0x7fcd0019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd00107aa0 0x7fcd001a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010da40 con 0x7fcd00105170 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010d8c0 con 0x7fcd00107aa0 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.622+0000 7fcd07000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010dbc0 con 0x7fcd00102840 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00102840 0x7fcd0019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00102840 0x7fcd0019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:32854/0 (socket says 172.21.15.46:32854) 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 -- 172.21.15.46:0/1641928627 learned_addr learned my addr 172.21.15.46:0/1641928627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05200640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00105170 0x7fcd0019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd06600640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd00107aa0 0x7fcd001a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 -- 172.21.15.46:0/1641928627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd00107aa0 msgr2=0x7fcd001a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd00107aa0 0x7fcd001a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 -- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00105170 msgr2=0x7fcd0019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00105170 0x7fcd0019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 -- 172.21.15.46:0/1641928627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd001a1f10 con 0x7fcd00102840 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.623+0000 7fcd05c00640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00102840 0x7fcd0019a2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9f0 tx=0x7fccf400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.624+0000 7fccfe000640 1 -- 172.21.15.46:0/1641928627 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400cd80 con 0x7fcd00102840 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.624+0000 7fccfe000640 1 -- 172.21.15.46:0/1641928627 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf4010910 con 0x7fcd00102840 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.624+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001a2200 con 0x7fcd00102840 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.624+0000 7fccfe000640 1 -- 172.21.15.46:0/1641928627 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf40188f0 con 0x7fcd00102840 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.624+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001a2750 con 0x7fcd00102840 2024-04-17T07:57:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.625+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd00109590 con 0x7fcd00102840 2024-04-17T07:57:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.627+0000 7fccfe000640 1 -- 172.21.15.46:0/1641928627 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fccf4010ac0 con 0x7fcd00102840 2024-04-17T07:57:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.628+0000 7fccfe000640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcce80a3bb0 0x7fcce80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.629+0000 7fccfe000640 1 -- 172.21.15.46:0/1641928627 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fccf40d0bd0 con 0x7fcd00102840 2024-04-17T07:57:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.629+0000 7fcd05200640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcce80a3bb0 0x7fcce80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.629+0000 7fcd05200640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcce80a3bb0 0x7fcce80a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fccec0095f0 tx=0x7fccec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.633+0000 7fccfe000640 1 -- 172.21.15.46:0/1641928627 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fccf409d220 con 0x7fcd00102840 2024-04-17T07:57:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.841+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcd00062b90 con 0x7fcd00102840 2024-04-17T07:57:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.842+0000 7fccfe000640 1 -- 172.21.15.46:0/1641928627 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fccf409cb20 con 0x7fcd00102840 2024-04-17T07:57:52.841 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcce80a3bb0 msgr2=0x7fcce80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcce80a3bb0 0x7fcce80a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fccec0095f0 tx=0x7fccec005f90 comp rx=0 tx=0).stop 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00102840 msgr2=0x7fcd0019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00102840 0x7fcd0019a2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9f0 tx=0x7fccf400eeb0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 shutdown_connections 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcce80a3bb0 0x7fcce80a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd00107aa0 0x7fcd001a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00105170 0x7fcd0019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 --2- 172.21.15.46:0/1641928627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd00102840 0x7fcd0019a2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 >> 172.21.15.46:0/1641928627 conn(0x7fcd000fc250 msgr2=0x7fcd00101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 shutdown_connections 2024-04-17T07:57:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:52.845+0000 7fcd07000640 1 -- 172.21.15.46:0/1641928627 wait complete. 2024-04-17T07:57:52.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 -- 172.21.15.46:0/2327854620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201052b0 msgr2=0x7f3420105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 --2- 172.21.15.46:0/2327854620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201052b0 0x7f3420105730 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3408009ca0 tx=0x7f340802c100 comp rx=0 tx=0).stop 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 -- 172.21.15.46:0/2327854620 shutdown_connections 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 --2- 172.21.15.46:0/2327854620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420105c70 0x7f342010c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 --2- 172.21.15.46:0/2327854620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201052b0 0x7f3420105730 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 --2- 172.21.15.46:0/2327854620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34201040c0 0x7f34201044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 -- 172.21.15.46:0/2327854620 >> 172.21.15.46:0/2327854620 conn(0x7f34200ff630 msgr2=0x7f3420101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 -- 172.21.15.46:0/2327854620 shutdown_connections 2024-04-17T07:57:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.080+0000 7f3424a00640 1 -- 172.21.15.46:0/2327854620 wait complete. 2024-04-17T07:57:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.081+0000 7f3424a00640 1 Processor -- start 2024-04-17T07:57:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3424a00640 1 -- start start 2024-04-17T07:57:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3424a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201040c0 0x7f342019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3424a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34201052b0 0x7f342019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3424a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3420105c70 0x7f34201a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3424a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f342010fd50 con 0x7f34201040c0 2024-04-17T07:57:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3424a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f342010fbd0 con 0x7f34201052b0 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3424a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f342010fed0 con 0x7f3420105c70 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f341f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201040c0 0x7f342019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f341f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201040c0 0x7f342019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38396/0 (socket says 172.21.15.46:38396) 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f341f400640 1 -- 172.21.15.46:0/3938950603 learned_addr learned my addr 172.21.15.46:0/3938950603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f3416a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34201052b0 0x7f342019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.082+0000 7f341fe00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3420105c70 0x7f34201a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.083+0000 7f3416a00640 1 -- 172.21.15.46:0/3938950603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3420105c70 msgr2=0x7f34201a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.083+0000 7f3416a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3420105c70 0x7f34201a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.083+0000 7f3416a00640 1 -- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201040c0 msgr2=0x7f342019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.083+0000 7f3416a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201040c0 0x7f342019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.083+0000 7f3416a00640 1 -- 172.21.15.46:0/3938950603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34201a1fa0 con 0x7f34201052b0 2024-04-17T07:57:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.083+0000 7f341fe00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3420105c70 0x7f34201a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:57:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.083+0000 7f3416a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34201052b0 0x7f342019a8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3408009bf0 tx=0x7f3408035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.084+0000 7f341cc00640 1 -- 172.21.15.46:0/3938950603 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408039040 con 0x7f34201052b0 2024-04-17T07:57:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.084+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34201a2230 con 0x7f34201052b0 2024-04-17T07:57:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.084+0000 7f341cc00640 1 -- 172.21.15.46:0/3938950603 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3408003970 con 0x7f34201052b0 2024-04-17T07:57:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.084+0000 7f341cc00640 1 -- 172.21.15.46:0/3938950603 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408004360 con 0x7f34201052b0 2024-04-17T07:57:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.084+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201a2740 con 0x7f34201052b0 2024-04-17T07:57:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.085+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342010b460 con 0x7f34201052b0 2024-04-17T07:57:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.087+0000 7f341cc00640 1 -- 172.21.15.46:0/3938950603 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3408046030 con 0x7f34201052b0 2024-04-17T07:57:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.089+0000 7f341cc00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34040a3ba0 0x7f34040a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.089+0000 7f341cc00640 1 -- 172.21.15.46:0/3938950603 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f34080f2490 con 0x7f34201052b0 2024-04-17T07:57:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.089+0000 7f341f400640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34040a3ba0 0x7f34040a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.090+0000 7f341f400640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34040a3ba0 0x7f34040a6060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3410005fd0 tx=0x7f341000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.093+0000 7f341cc00640 1 -- 172.21.15.46:0/3938950603 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f340804d060 con 0x7f34201052b0 2024-04-17T07:57:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.343+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3420062b60 con 0x7f34201052b0 2024-04-17T07:57:53.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.344+0000 7f341cc00640 1 -- 172.21.15.46:0/3938950603 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f34080d2e20 con 0x7f34201052b0 2024-04-17T07:57:53.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:53.343 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:54.519826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24556,24542]},"id":1}]} 2024-04-17T07:57:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-04-17T07:57:53.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34040a3ba0 msgr2=0x7f34040a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34040a3ba0 0x7f34040a6060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3410005fd0 tx=0x7f341000b4a0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34201052b0 msgr2=0x7f342019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34201052b0 0x7f342019a8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3408009bf0 tx=0x7f3408035d10 comp rx=0 tx=0).stop 2024-04-17T07:57:53.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 shutdown_connections 2024-04-17T07:57:53.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34040a3ba0 0x7f34040a6060 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3420105c70 0x7f34201a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34201052b0 0x7f342019a8c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 --2- 172.21.15.46:0/3938950603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34201040c0 0x7f342019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 >> 172.21.15.46:0/3938950603 conn(0x7f34200ff630 msgr2=0x7f3420108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 shutdown_connections 2024-04-17T07:57:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.347+0000 7f3424a00640 1 -- 172.21.15.46:0/3938950603 wait complete. 2024-04-17T07:57:53.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:57:53.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:53 smithi046 ceph-mon[32038]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:53.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:53 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2788543567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:57:53.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:53 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1641928627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:57:53.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:53 smithi053 ceph-mon[36437]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:53.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:53 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2788543567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:57:53.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:53 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1641928627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:57:53.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:53 smithi062 ceph-mon[36507]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:57:53.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:53 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2788543567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:57:53.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:53 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1641928627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:57:53.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.576+0000 7f3eee200640 1 -- 172.21.15.46:0/1082792395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee810a060 msgr2=0x7f3ee81108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.576+0000 7f3eee200640 1 --2- 172.21.15.46:0/1082792395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee810a060 0x7f3ee81108f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009ca0 tx=0x7f3ee002c100 comp rx=0 tx=0).stop 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.576+0000 7f3eee200640 1 -- 172.21.15.46:0/1082792395 shutdown_connections 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.576+0000 7f3eee200640 1 --2- 172.21.15.46:0/1082792395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee810a060 0x7f3ee81108f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.576+0000 7f3eee200640 1 --2- 172.21.15.46:0/1082792395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee81096a0 0x7f3ee8109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.576+0000 7f3eee200640 1 --2- 172.21.15.46:0/1082792395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 0x7f3ee8108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.576+0000 7f3eee200640 1 -- 172.21.15.46:0/1082792395 >> 172.21.15.46:0/1082792395 conn(0x7f3ee8077450 msgr2=0x7f3ee8079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.577+0000 7f3eee200640 1 -- 172.21.15.46:0/1082792395 shutdown_connections 2024-04-17T07:57:53.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.577+0000 7f3eee200640 1 -- 172.21.15.46:0/1082792395 wait complete. 2024-04-17T07:57:53.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.577+0000 7f3eee200640 1 Processor -- start 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eee200640 1 -- start start 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eee200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 0x7f3ee819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eee200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee81096a0 0x7f3ee819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eee200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810a060 0x7f3ee81a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eee200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8114140 con 0x7f3ee810a060 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eee200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8113fc0 con 0x7f3ee8108500 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eee200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee81142c0 con 0x7f3ee81096a0 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eece00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 0x7f3ee819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3ee7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee81096a0 0x7f3ee819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eece00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 0x7f3ee819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51506/0 (socket says 172.21.15.46:51506) 2024-04-17T07:57:53.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eece00640 1 -- 172.21.15.46:0/3733832752 learned_addr learned my addr 172.21.15.46:0/3733832752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.578+0000 7f3eed800640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810a060 0x7f3ee81a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee7e00640 1 -- 172.21.15.46:0/3733832752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 msgr2=0x7f3ee819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee7e00640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 0x7f3ee819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee7e00640 1 -- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810a060 msgr2=0x7f3ee81a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee7e00640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810a060 0x7f3ee81a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee7e00640 1 -- 172.21.15.46:0/3733832752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee81a6350 con 0x7f3ee81096a0 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3eece00640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 0x7f3ee819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee7e00640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee81096a0 0x7f3ee819ec70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800e9f0 tx=0x7f3ed800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee5600640 1 -- 172.21.15.46:0/3733832752 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed800ccc0 con 0x7f3ee81096a0 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee5600640 1 -- 172.21.15.46:0/3733832752 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed8010910 con 0x7f3ee81096a0 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee81a6640 con 0x7f3ee81096a0 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3ee5600640 1 -- 172.21.15.46:0/3733832752 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8018960 con 0x7f3ee81096a0 2024-04-17T07:57:53.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.579+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee81a6be0 con 0x7f3ee81096a0 2024-04-17T07:57:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.580+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee810f700 con 0x7f3ee81096a0 2024-04-17T07:57:53.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.582+0000 7f3ee5600640 1 -- 172.21.15.46:0/3733832752 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3ed8010ac0 con 0x7f3ee81096a0 2024-04-17T07:57:53.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.584+0000 7f3ee5600640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3ed00a3c20 0x7f3ed00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.584+0000 7f3ee5600640 1 -- 172.21.15.46:0/3733832752 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f3ed80d0af0 con 0x7f3ee81096a0 2024-04-17T07:57:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.584+0000 7f3eece00640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3ed00a3c20 0x7f3ed00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:53.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.585+0000 7f3eece00640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3ed00a3c20 0x7f3ed00a60e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0058c0 tx=0x7f3edc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.588+0000 7f3ee5600640 1 -- 172.21.15.46:0/3733832752 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3ed809ca90 con 0x7f3ee81096a0 2024-04-17T07:57:53.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.798+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ee8062b60 con 0x7f3ee81096a0 2024-04-17T07:57:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.800+0000 7f3ee5600640 1 -- 172.21.15.46:0/3733832752 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7f3ed80a0d90 con 0x7f3ee81096a0 2024-04-17T07:57:53.800 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:53.800 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:57:53.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3ed00a3c20 msgr2=0x7f3ed00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3ed00a3c20 0x7f3ed00a60e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0058c0 tx=0x7f3edc005f90 comp rx=0 tx=0).stop 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee81096a0 msgr2=0x7f3ee819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee81096a0 0x7f3ee819ec70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800e9f0 tx=0x7f3ed800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 shutdown_connections 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3ed00a3c20 0x7f3ed00a60e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee810a060 0x7f3ee81a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee81096a0 0x7f3ee819ec70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 --2- 172.21.15.46:0/3733832752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ee8108500 0x7f3ee819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 >> 172.21.15.46:0/3733832752 conn(0x7f3ee8077450 msgr2=0x7f3ee810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.803+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 shutdown_connections 2024-04-17T07:57:53.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:53.804+0000 7f3eee200640 1 -- 172.21.15.46:0/3733832752 wait complete. 2024-04-17T07:57:53.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:57:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 -- 172.21.15.46:0/3271843544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102910 msgr2=0x7f59ac104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 --2- 172.21.15.46:0/3271843544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102910 0x7f59ac104d00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f599c009ca0 tx=0x7f599c02c100 comp rx=0 tx=0).stop 2024-04-17T07:57:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 -- 172.21.15.46:0/3271843544 shutdown_connections 2024-04-17T07:57:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 --2- 172.21.15.46:0/3271843544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac107b70 0x7f59ac109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 --2- 172.21.15.46:0/3271843544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59ac105240 0x7f59ac107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 --2- 172.21.15.46:0/3271843544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac102910 0x7f59ac104d00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 -- 172.21.15.46:0/3271843544 >> 172.21.15.46:0/3271843544 conn(0x7f59ac0fc340 msgr2=0x7f59ac0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 -- 172.21.15.46:0/3271843544 shutdown_connections 2024-04-17T07:57:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.088+0000 7f59abe00640 1 -- 172.21.15.46:0/3271843544 wait complete. 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 Processor -- start 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 -- start start 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac102910 0x7f59ac19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59ac105240 0x7f59ac19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac107b70 0x7f59ac1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10db30 con 0x7f59ac107b70 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10d9b0 con 0x7f59ac105240 2024-04-17T07:57:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.089+0000 7f59abe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10dcb0 con 0x7f59ac102910 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac107b70 0x7f59ac1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac102910 0x7f59ac19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac102910 0x7f59ac19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:32904/0 (socket says 172.21.15.46:32904) 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 -- 172.21.15.46:0/4084040478 learned_addr learned my addr 172.21.15.46:0/4084040478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59a2000640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59ac105240 0x7f59ac19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 -- 172.21.15.46:0/4084040478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59ac105240 msgr2=0x7f59ac19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59ac105240 0x7f59ac19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 -- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac107b70 msgr2=0x7f59ac1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac107b70 0x7f59ac1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.090+0000 7f59aaa00640 1 -- 172.21.15.46:0/4084040478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac1a1fc0 con 0x7f59ac102910 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.091+0000 7f59aaa00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac102910 0x7f59ac19a3a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f599c0325f0 tx=0x7f599c035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.091+0000 7f59b0a00640 1 -- 172.21.15.46:0/4084040478 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c039040 con 0x7f59ac102910 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.091+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac1a2250 con 0x7f59ac102910 2024-04-17T07:57:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.091+0000 7f59b0a00640 1 -- 172.21.15.46:0/4084040478 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f599c044070 con 0x7f59ac102910 2024-04-17T07:57:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.091+0000 7f59b0a00640 1 -- 172.21.15.46:0/4084040478 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0044d0 con 0x7f59ac102910 2024-04-17T07:57:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.091+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac1a2710 con 0x7f59ac102910 2024-04-17T07:57:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.092+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ac1096c0 con 0x7f59ac102910 2024-04-17T07:57:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.094+0000 7f59b0a00640 1 -- 172.21.15.46:0/4084040478 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f599c004670 con 0x7f59ac102910 2024-04-17T07:57:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.096+0000 7f59b0a00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f59900a3ba0 0x7f59900a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.096+0000 7f59b0a00640 1 -- 172.21.15.46:0/4084040478 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f599c0f3170 con 0x7f59ac102910 2024-04-17T07:57:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.096+0000 7f59a2000640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f59900a3ba0 0x7f59900a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.097+0000 7f59a2000640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f59900a3ba0 0x7f59900a6060 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59ac19b8b0 tx=0x7f5994005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.098+0000 7f59b0a00640 1 -- 172.21.15.46:0/4084040478 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f599c0bf880 con 0x7f59ac102910 2024-04-17T07:57:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.339+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59ac02d5d0 con 0x7f59ac102910 2024-04-17T07:57:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.340+0000 7f59b0a00640 1 -- 172.21.15.46:0/4084040478 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f599c0bf180 con 0x7f59ac102910 2024-04-17T07:57:54.340 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:57:54.340 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:54.519826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24556,24542]},"id":1}]} 2024-04-17T07:57:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-04-17T07:57:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f59900a3ba0 msgr2=0x7f59900a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f59900a3ba0 0x7f59900a6060 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59ac19b8b0 tx=0x7f5994005f90 comp rx=0 tx=0).stop 2024-04-17T07:57:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac102910 msgr2=0x7f59ac19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac102910 0x7f59ac19a3a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f599c0325f0 tx=0x7f599c035d10 comp rx=0 tx=0).stop 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 shutdown_connections 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f59900a3ba0 0x7f59900a6060 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59ac107b70 0x7f59ac1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59ac105240 0x7f59ac19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 --2- 172.21.15.46:0/4084040478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59ac102910 0x7f59ac19a3a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 >> 172.21.15.46:0/4084040478 conn(0x7f59ac0fc340 msgr2=0x7f59ac1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 shutdown_connections 2024-04-17T07:57:54.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.343+0000 7f59abe00640 1 -- 172.21.15.46:0/4084040478 wait complete. 2024-04-17T07:57:54.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T07:55:26.016968+0000', 'modified': '2024-04-17T07:55:54.519826+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24596, 'mds_1': 24556, 'mds_2': 24542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/2191814520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2191814520}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2191814520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/956269488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 956269488}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 956269488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/665634381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 665634381}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 665634381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.53:6837/2183592848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 2183592848}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 2183592848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24556': {'gid': 24556, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.46:6839/137300158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 137300158}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 137300158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/262156089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 262156089}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 262156089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24596, 24556, 24542]} 2024-04-17T07:57:54.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-17T07:57:54.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-17T07:57:54.445 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:54 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3938950603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:54.445 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:54 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3733832752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:54.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:54 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3938950603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:54.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:54 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3733832752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:54.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:54 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3938950603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:54.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:54 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3733832752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:57:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.590+0000 7fe08be00640 1 -- 172.21.15.46:0/2831580596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c105b90 msgr2=0x7fe08c10c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.590+0000 7fe08be00640 1 --2- 172.21.15.46:0/2831580596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c105b90 0x7fe08c10c420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe084009860 tx=0x7fe08402bb70 comp rx=0 tx=0).stop 2024-04-17T07:57:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.591+0000 7fe08be00640 1 -- 172.21.15.46:0/2831580596 shutdown_connections 2024-04-17T07:57:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.591+0000 7fe08be00640 1 --2- 172.21.15.46:0/2831580596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c105b90 0x7fe08c10c420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.591+0000 7fe08be00640 1 --2- 172.21.15.46:0/2831580596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe08c1051d0 0x7fe08c105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.591+0000 7fe08be00640 1 --2- 172.21.15.46:0/2831580596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c103fe0 0x7fe08c104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.591+0000 7fe08be00640 1 -- 172.21.15.46:0/2831580596 >> 172.21.15.46:0/2831580596 conn(0x7fe08c0ff570 msgr2=0x7fe08c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.591+0000 7fe08be00640 1 -- 172.21.15.46:0/2831580596 shutdown_connections 2024-04-17T07:57:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.591+0000 7fe08be00640 1 -- 172.21.15.46:0/2831580596 wait complete. 2024-04-17T07:57:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.592+0000 7fe08be00640 1 Processor -- start 2024-04-17T07:57:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.592+0000 7fe08be00640 1 -- start start 2024-04-17T07:57:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.592+0000 7fe08be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103fe0 0x7fe08c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.592+0000 7fe08be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c1051d0 0x7fe08c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.592+0000 7fe08be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe08c105b90 0x7fe08c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c10fc70 con 0x7fe08c103fe0 2024-04-17T07:57:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08be00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c10faf0 con 0x7fe08c1051d0 2024-04-17T07:57:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c10fdf0 con 0x7fe08c105b90 2024-04-17T07:57:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103fe0 0x7fe08c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103fe0 0x7fe08c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38432/0 (socket says 172.21.15.46:38432) 2024-04-17T07:57:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 -- 172.21.15.46:0/2636475087 learned_addr learned my addr 172.21.15.46:0/2636475087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08b400640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe08c105b90 0x7fe08c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08a000640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c1051d0 0x7fe08c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 -- 172.21.15.46:0/2636475087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe08c105b90 msgr2=0x7fe08c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe08c105b90 0x7fe08c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 -- 172.21.15.46:0/2636475087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c1051d0 msgr2=0x7fe08c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c1051d0 0x7fe08c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.593+0000 7fe08aa00640 1 -- 172.21.15.46:0/2636475087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c1a1f10 con 0x7fe08c103fe0 2024-04-17T07:57:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.594+0000 7fe08aa00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103fe0 0x7fe08c19a2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e9c0 tx=0x7fe07c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.594+0000 7fe090a00640 1 -- 172.21.15.46:0/2636475087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00cde0 con 0x7fe08c103fe0 2024-04-17T07:57:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.594+0000 7fe090a00640 1 -- 172.21.15.46:0/2636475087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe07c010910 con 0x7fe08c103fe0 2024-04-17T07:57:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.594+0000 7fe090a00640 1 -- 172.21.15.46:0/2636475087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c018a00 con 0x7fe08c103fe0 2024-04-17T07:57:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.594+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c1a2200 con 0x7fe08c103fe0 2024-04-17T07:57:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.594+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c1a2700 con 0x7fe08c103fe0 2024-04-17T07:57:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 Processor -- start 2024-04-17T07:57:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 -- start start 2024-04-17T07:57:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e27f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe090a00640 1 -- 172.21.15.46:0/2636475087 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe07c01f020 con 0x7fe08c103fe0 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e2fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556c9d8e980 con 0x5556c9e27af0 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5556c9d35220 con 0x5556c9e2d650 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.598+0000 7fe094b8b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556c9d8a940 con 0x5556c9e284e0 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e27f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e2cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:32934/0 (socket says 172.21.15.46:32934) 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 -- 172.21.15.46:0/664857835 learned_addr learned my addr 172.21.15.46:0/664857835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08a000640 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e2fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 -- 172.21.15.46:0/664857835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 msgr2=0x5556c9e2fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e2fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 -- 172.21.15.46:0/664857835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 msgr2=0x5556c9e27f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e27f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08aa00640 1 -- 172.21.15.46:0/664857835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c9aa12d0 con 0x5556c9e284e0 2024-04-17T07:57:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.599+0000 7fe08a000640 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e2fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:57:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.600+0000 7fe08aa00640 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e2cfe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0d6c20 tx=0x7fe07c0d6d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed6927446f812cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.600+0000 7fe077e00640 1 -- 172.21.15.46:0/664857835 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0df070 con 0x5556c9e284e0 2024-04-17T07:57:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.600+0000 7fe090a00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe0700a3c30 0x7fe0700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.600+0000 7fe077e00640 1 -- 172.21.15.46:0/664857835 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe07c09cf30 con 0x5556c9e284e0 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.600+0000 7fe077e00640 1 -- 172.21.15.46:0/664857835 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0618b0 con 0x5556c9e284e0 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.600+0000 7fe090a00640 1 -- 172.21.15.46:0/2636475087 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe07c0d0940 con 0x7fe08c103fe0 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.600+0000 7fe08b400640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe0700a3c30 0x7fe0700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 -- 172.21.15.46:0/664857835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 msgr2=0x5556c9e2cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e2cfe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0d6c20 tx=0x7fe07c0d6d10 comp rx=0 tx=0).stop 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe08b400640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe0700a3c30 0x7fe0700a60f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe084034a50 tx=0x7fe0840349e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 -- 172.21.15.46:0/664857835 shutdown_connections 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e2fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e2cfe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 --2- 172.21.15.46:0/664857835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e27f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 -- 172.21.15.46:0/664857835 >> 172.21.15.46:0/664857835 conn(0x5556c9e23060 msgr2=0x5556c9e23460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 -- 172.21.15.46:0/664857835 shutdown_connections 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.601+0000 7fe094b8b740 1 -- 172.21.15.46:0/664857835 wait complete. 2024-04-17T07:57:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.602+0000 7fe094b8b740 1 -- start start 2024-04-17T07:57:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.602+0000 7fe094b8b740 1 Processor -- start 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e32510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe094b8b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e30600 con 0x5556c9e27af0 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe094b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5556c9d48590 con 0x5556c9e2d650 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe094b8b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e30780 con 0x5556c9e284e0 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe094b8b740 10 client.0 _ll_register_callbacks cb 0x5556c9aa0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe08aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe08a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e32510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe08aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e31fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38456/0 (socket says 172.21.15.46:38456) 2024-04-17T07:57:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe08b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.603+0000 7fe08aa00640 1 -- 172.21.15.46:0/204945773 learned_addr learned my addr 172.21.15.46:0/204945773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.604+0000 7fe08b400640 1 -- 172.21.15.46:0/204945773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 msgr2=0x5556c9e32510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.604+0000 7fe08b400640 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e32510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.604+0000 7fe08b400640 1 -- 172.21.15.46:0/204945773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 msgr2=0x5556c9e31fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.604+0000 7fe08b400640 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e31fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.604+0000 7fe08b400640 1 -- 172.21.15.46:0/204945773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c9b6b6b0 con 0x5556c9e2d650 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.604+0000 7fe08aa00640 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e31fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe08b400640 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e39550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe08402c0b0 tx=0x7fe08402bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b92fded30705a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe074c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe074c00640 1 -- 172.21.15.46:0/204945773 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08404f020 con 0x5556c9e2d650 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe07c09cf30 con 0x5556c9e2d650 2024-04-17T07:57:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe074c00640 1 -- 172.21.15.46:0/204945773 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe08402c710 con 0x5556c9e2d650 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe074c00640 1 -- 172.21.15.46:0/204945773 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe08405a050 con 0x5556c9e2d650 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe094b8b740 10 client.25193 fetch_fsmap learned FSMap version 23 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.605+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5556c9e3c980 con 0x5556c9e2d650 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe074c00640 1 -- 172.21.15.46:0/204945773 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe07c09cf30 con 0x5556c9e2d650 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe074c00640 1 -- 172.21.15.46:0/204945773 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7fe0840035f0 con 0x5556c9e2d650 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe094b8b740 10 client.25193 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe094b8b740 10 client.25193 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe094b8b740 20 client.25193 populate_metadata read hostname 'smithi046' 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe094b8b740 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9e3ef80 0x5556c9e403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe094b8b740 4 client.25193 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556c9d9c3a0 con 0x5556c9e3ef80 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.606+0000 7fe08aa00640 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9e3ef80 0x5556c9e403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.607+0000 7fe08aa00640 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9e3ef80 0x5556c9e403f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd536d1aeaf52a32 server_cookie=6ec4c7d63725ffb9 in_seq=0 out_seq=0 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.607+0000 7fe074c00640 10 client.25193 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe074c00640 1 -- 172.21.15.46:0/204945773 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556c9d9c3a0 con 0x5556c9e3ef80 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe074c00640 10 client.25193 handle_command_reply: tid=0 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 client.25193 shutdown 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 msgr2=0x5556c9e39550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e39550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe08402c0b0 tx=0x7fe08402bb70 comp rx=0 tx=0).stop 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 shutdown_connections 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9e3ef80 0x5556c9e403f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e2d650 0x5556c9e39550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e284e0 0x5556c9e32510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe074c00640 0 client.25193 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 --2- 172.21.15.46:0/204945773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e27af0 0x5556c9e31fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 >> 172.21.15.46:0/204945773 conn(0x5556c9e23060 msgr2=0x5556c9e223a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 shutdown_connections 2024-04-17T07:57:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 1 -- 172.21.15.46:0/204945773 wait complete. 2024-04-17T07:57:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.608+0000 7fe094b8b740 20 client.25193 trim_cache size 0 max 16384 2024-04-17T07:57:54.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 Processor -- start 2024-04-17T07:57:54.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 -- start start 2024-04-17T07:57:54.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e39510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e3e730 con 0x5556c9e284e0 2024-04-17T07:57:54.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e3ee00 con 0x5556c9e27af0 2024-04-17T07:57:54.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.630+0000 7fe094b8b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e3f4d0 con 0x5556c9e2d650 2024-04-17T07:57:54.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.631+0000 7fe08a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.631+0000 7fe08a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e39510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38458/0 (socket says 172.21.15.46:38458) 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.631+0000 7fe08a000640 1 -- 172.21.15.46:0/561997830 learned_addr learned my addr 172.21.15.46:0/561997830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.631+0000 7fe08b400640 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.631+0000 7fe08aa00640 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe08b400640 1 -- 172.21.15.46:0/561997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 msgr2=0x5556c9e403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe08b400640 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe08b400640 1 -- 172.21.15.46:0/561997830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 msgr2=0x5556c9e39510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe08b400640 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e39510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe08b400640 1 -- 172.21.15.46:0/561997830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c9e14900 con 0x5556c9e2d650 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe08b400640 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e21b10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0840057c0 tx=0x7fe084005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28cc1aa13ca0c2af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe077400640 1 -- 172.21.15.46:0/561997830 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08404f070 con 0x5556c9e2d650 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe077400640 1 -- 172.21.15.46:0/561997830 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe084063070 con 0x5556c9e2d650 2024-04-17T07:57:54.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.632+0000 7fe077400640 1 -- 172.21.15.46:0/561997830 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084039040 con 0x5556c9e2d650 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 -- 172.21.15.46:0/561997830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 msgr2=0x5556c9e21b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e21b10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0840057c0 tx=0x7fe084005970 comp rx=0 tx=0).stop 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 -- 172.21.15.46:0/561997830 shutdown_connections 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e21b10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e39510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 --2- 172.21.15.46:0/561997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 -- 172.21.15.46:0/561997830 >> 172.21.15.46:0/561997830 conn(0x5556c9e23060 msgr2=0x5556c9e23440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 -- 172.21.15.46:0/561997830 shutdown_connections 2024-04-17T07:57:54.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.633+0000 7fe094b8b740 1 -- 172.21.15.46:0/561997830 wait complete. 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 -- start start 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 Processor -- start 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e35460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e38970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e40270 con 0x5556c9e284e0 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e3fba0 con 0x5556c9e27af0 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556c9e40940 con 0x5556c9e2d650 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.634+0000 7fe094b8b740 10 client.0 _ll_register_callbacks cb 0x7fe078009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e359a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38462/0 (socket says 172.21.15.46:38462) 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 -- 172.21.15.46:0/3558160795 learned_addr learned my addr 172.21.15.46:0/3558160795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08b400640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e38970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08aa00640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e35460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 -- 172.21.15.46:0/3558160795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 msgr2=0x5556c9e38970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e38970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 -- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 msgr2=0x5556c9e35460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e35460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 -- 172.21.15.46:0/3558160795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c9e38f60 con 0x5556c9e284e0 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08b400640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e38970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe08a000640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e359a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe07800ee70 tx=0x7fe07800d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eec8899da47a212 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.635+0000 7fe074c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.636+0000 7fe074c00640 1 -- 172.21.15.46:0/3558160795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07801e070 con 0x5556c9e284e0 2024-04-17T07:57:54.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.636+0000 7fe074c00640 1 -- 172.21.15.46:0/3558160795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe078003b70 con 0x5556c9e284e0 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.636+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe084063070 con 0x5556c9e284e0 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.636+0000 7fe074c00640 1 -- 172.21.15.46:0/3558160795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe07802a050 con 0x5556c9e284e0 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.636+0000 7fe094b8b740 10 client.15066 fetch_fsmap learned FSMap version 23 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.636+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5556c9ea5750 con 0x5556c9e284e0 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.636+0000 7fe074c00640 1 -- 172.21.15.46:0/3558160795 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe084063070 con 0x5556c9e284e0 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe074c00640 1 -- 172.21.15.46:0/3558160795 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7fe07800b040 con 0x5556c9e284e0 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe094b8b740 10 client.15066 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe094b8b740 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe094b8b740 20 client.15066 populate_metadata read hostname 'smithi046' 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe094b8b740 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9eaa400 0x5556c9ea9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe094b8b740 4 client.15066 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "status"}) -- 0x5556c9d96210 con 0x5556c9eaa400 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.637+0000 7fe08aa00640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9eaa400 0x5556c9ea9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.638+0000 7fe074c00640 10 client.15066 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:57:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.638+0000 7fe08aa00640 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9eaa400 0x5556c9ea9110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ce6ffb48fa0ef37 server_cookie=3a88e54ff8eaf877 in_seq=0 out_seq=0 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.638+0000 7fe074c00640 1 -- 172.21.15.46:0/3558160795 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5556c9d96210 con 0x5556c9eaa400 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.638+0000 7fe074c00640 10 client.15066 handle_command_reply: tid=0 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.638+0000 7fe094b8b740 1 client.15066 shutdown 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.638+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 msgr2=0x5556c9e359a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.638+0000 7fe094b8b740 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e359a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe07800ee70 tx=0x7fe07800d7b0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 shutdown_connections 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5556c9eaa400 0x5556c9ea9110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556c9e2d650 0x5556c9e38970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556c9e284e0 0x5556c9e359a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 --2- 172.21.15.46:0/3558160795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556c9e27af0 0x5556c9e35460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 >> 172.21.15.46:0/3558160795 conn(0x5556c9e23060 msgr2=0x5556c9e262c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe074c00640 0 client.15066 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 shutdown_connections 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 1 -- 172.21.15.46:0/3558160795 wait complete. 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.639+0000 7fe094b8b740 20 client.15066 trim_cache size 0 max 16384 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24542, 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-17T07:57:54.643 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-17T07:57:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-17T07:57:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 120.919661405, 2024-04-17T07:57:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 23, 2024-04-17T07:57:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-04-17T07:57:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:57:54.644 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 169.65798617999999 2024-04-17T07:57:54.644 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe0700a3c30 msgr2=0x7fe0700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe0700a3c30 0x7fe0700a60f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe084034a50 tx=0x7fe0840349e0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103fe0 msgr2=0x7fe08c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103fe0 0x7fe08c19a2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e9c0 tx=0x7fe07c00ee80 comp rx=0 tx=0).stop 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 shutdown_connections 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fe0700a3c30 0x7fe0700a60f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe08c105b90 0x7fe08c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe08c1051d0 0x7fe08c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 --2- 172.21.15.46:0/2636475087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe08c103fe0 0x7fe08c19a2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 >> 172.21.15.46:0/2636475087 conn(0x7fe08c0ff570 msgr2=0x7fe08c108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 shutdown_connections 2024-04-17T07:57:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.640+0000 7fe08be00640 1 -- 172.21.15.46:0/2636475087 wait complete. 2024-04-17T07:57:54.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 -- 172.21.15.46:0/4276968801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 msgr2=0x7f028c105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 --2- 172.21.15.46:0/4276968801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 0x7f028c105820 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0278009900 tx=0x7f027802bc50 comp rx=0 tx=0).stop 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 -- 172.21.15.46:0/4276968801 shutdown_connections 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 --2- 172.21.15.46:0/4276968801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c105d60 0x7f028c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 --2- 172.21.15.46:0/4276968801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 0x7f028c105820 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 --2- 172.21.15.46:0/4276968801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 0x7f028c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 -- 172.21.15.46:0/4276968801 >> 172.21.15.46:0/4276968801 conn(0x7f028c0ff720 msgr2=0x7f028c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 -- 172.21.15.46:0/4276968801 shutdown_connections 2024-04-17T07:57:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.887+0000 7f0291800640 1 -- 172.21.15.46:0/4276968801 wait complete. 2024-04-17T07:57:54.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.888+0000 7f0291800640 1 Processor -- start 2024-04-17T07:57:54.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.888+0000 7f0291800640 1 -- start start 2024-04-17T07:57:54.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f0291800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 0x7f028c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f0291800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 0x7f028c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f0291800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c105d60 0x7f028c1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f0291800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10fe90 con 0x7f028c1053a0 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f0291800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10fd10 con 0x7f028c1041b0 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f0291800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f028c110010 con 0x7f028c105d60 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 0x7f028c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 0x7f028c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 0x7f028c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51630/0 (socket says 172.21.15.46:51630) 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 -- 172.21.15.46:0/1783369397 learned_addr learned my addr 172.21.15.46:0/1783369397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028b400640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c105d60 0x7f028c1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 -- 172.21.15.46:0/1783369397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c105d60 msgr2=0x7f028c1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c105d60 0x7f028c1a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 -- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 msgr2=0x7f028c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 0x7f028c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.889+0000 7f028aa00640 1 -- 172.21.15.46:0/1783369397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c1a2120 con 0x7f028c1041b0 2024-04-17T07:57:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.890+0000 7f028aa00640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 0x7f028c19a500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9c0 tx=0x7f027c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.890+0000 7f0283400640 1 -- 172.21.15.46:0/1783369397 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00cde0 con 0x7f028c1041b0 2024-04-17T07:57:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.890+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c1a2410 con 0x7f028c1041b0 2024-04-17T07:57:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.890+0000 7f0283400640 1 -- 172.21.15.46:0/1783369397 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c010910 con 0x7f028c1041b0 2024-04-17T07:57:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.890+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1a2980 con 0x7f028c1041b0 2024-04-17T07:57:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.890+0000 7f0283400640 1 -- 172.21.15.46:0/1783369397 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c018980 con 0x7f028c1041b0 2024-04-17T07:57:54.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.893+0000 7f02945e1740 1 Processor -- start 2024-04-17T07:57:54.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f02945e1740 1 -- start start 2024-04-17T07:57:54.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f0283400640 1 -- 172.21.15.46:0/1783369397 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f027c01f020 con 0x7f028c1041b0 2024-04-17T07:57:54.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f02945e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1212d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f02945e1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad1217dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f02945e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad121a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f02945e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ad1179980 con 0x556ad1218440 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f02945e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556ad1120220 con 0x556ad12128e0 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f02945e1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ad1175940 con 0x556ad12132d0 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f028a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad121a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f028b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1212d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f028a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad121a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38482/0 (socket says 172.21.15.46:38482) 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f028a000640 1 -- 172.21.15.46:0/111446746 learned_addr learned my addr 172.21.15.46:0/111446746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.894+0000 7f028b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1212d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51640/0 (socket says 172.21.15.46:51640) 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f028aa00640 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad1217dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f028b400640 1 -- 172.21.15.46:0/111446746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 msgr2=0x556ad1217dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f028b400640 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad1217dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f028b400640 1 -- 172.21.15.46:0/111446746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 msgr2=0x556ad121a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f028b400640 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad121a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f028b400640 1 -- 172.21.15.46:0/111446746 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad0e8c2d0 con 0x556ad12128e0 2024-04-17T07:57:54.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f028b400640 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1212d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0284009ed0 tx=0x7f02840107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f39b08b8c0614bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.895+0000 7f0283400640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f02700a3bb0 0x7f02700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f0277e00640 1 -- 172.21.15.46:0/111446746 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284003b70 con 0x556ad12128e0 2024-04-17T07:57:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f0277e00640 1 -- 172.21.15.46:0/111446746 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0284015070 con 0x556ad12128e0 2024-04-17T07:57:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f0277e00640 1 -- 172.21.15.46:0/111446746 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284011870 con 0x556ad12128e0 2024-04-17T07:57:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f0283400640 1 -- 172.21.15.46:0/1783369397 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f027c0d0ae0 con 0x7f028c1041b0 2024-04-17T07:57:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f028aa00640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f02700a3bb0 0x7f02700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f02945e1740 1 -- 172.21.15.46:0/111446746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 msgr2=0x556ad1212d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f02945e1740 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1212d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0284009ed0 tx=0x7f02840107a0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f028aa00640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f02700a3bb0 0x7f02700a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f027c098e30 tx=0x7f027c00efb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f02945e1740 1 -- 172.21.15.46:0/111446746 shutdown_connections 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f02945e1740 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad121a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f02945e1740 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad1217dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f02945e1740 1 --2- 172.21.15.46:0/111446746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1212d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.896+0000 7f02945e1740 1 -- 172.21.15.46:0/111446746 >> 172.21.15.46:0/111446746 conn(0x556ad120de00 msgr2=0x556ad120e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.897+0000 7f02945e1740 1 -- 172.21.15.46:0/111446746 shutdown_connections 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.897+0000 7f02945e1740 1 -- 172.21.15.46:0/111446746 wait complete. 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.897+0000 7f02945e1740 1 -- start start 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.897+0000 7f02945e1740 1 Processor -- start 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f02945e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad121ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f02945e1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad121d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f02945e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad12243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f02945e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ad121b3f0 con 0x556ad1218440 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f02945e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556ad1133590 con 0x556ad12128e0 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f02945e1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ad121b570 con 0x556ad12132d0 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f02945e1740 10 client.0 _ll_register_callbacks cb 0x556ad0e8bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f028b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad12243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f028b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad12243a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38494/0 (socket says 172.21.15.46:38494) 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f028a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad121d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f028b400640 1 -- 172.21.15.46:0/4000471716 learned_addr learned my addr 172.21.15.46:0/4000471716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.898+0000 7f028b400640 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad121ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f028a000640 1 -- 172.21.15.46:0/4000471716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 msgr2=0x556ad121ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f028a000640 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad121ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f028a000640 1 -- 172.21.15.46:0/4000471716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 msgr2=0x556ad12243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f028a000640 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad12243a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f028a000640 1 -- 172.21.15.46:0/4000471716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad0f566b0 con 0x556ad12132d0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f028b400640 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad121ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f028a000640 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad121d360 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0278006010 tx=0x7f0278009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddd49933ea0e125 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f0274c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f0274c00640 1 -- 172.21.15.46:0/4000471716 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278005d50 con 0x556ad12132d0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0284015070 con 0x556ad12132d0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.899+0000 7f0274c00640 1 -- 172.21.15.46:0/4000471716 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f02780048c0 con 0x556ad12132d0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.900+0000 7f0274c00640 1 -- 172.21.15.46:0/4000471716 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0278004ac0 con 0x556ad12132d0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.900+0000 7f02945e1740 10 client.25195 fetch_fsmap learned FSMap version 23 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.900+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556ad12277d0 con 0x556ad12132d0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.900+0000 7f0274c00640 1 -- 172.21.15.46:0/4000471716 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0284015070 con 0x556ad12132d0 2024-04-17T07:57:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f0274c00640 1 -- 172.21.15.46:0/4000471716 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7f0278046070 con 0x556ad12132d0 2024-04-17T07:57:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f02945e1740 10 client.25195 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:57:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f02945e1740 10 client.25195 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-17T07:57:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f02945e1740 20 client.25195 populate_metadata read hostname 'smithi046' 2024-04-17T07:57:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f02945e1740 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1229e10 0x556ad122b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f02945e1740 4 client.25195 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:57:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad11873a0 con 0x556ad1229e10 2024-04-17T07:57:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.901+0000 7f028b400640 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1229e10 0x556ad122b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.902+0000 7f028b400640 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1229e10 0x556ad122b280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4115f9f58f442ad3 server_cookie=c7bae7e060b2041c in_seq=0 out_seq=0 2024-04-17T07:57:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.902+0000 7f0274c00640 10 client.25195 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:57:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.903+0000 7f0274c00640 1 -- 172.21.15.46:0/4000471716 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ad11873a0 con 0x556ad1229e10 2024-04-17T07:57:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.903+0000 7f0274c00640 10 client.25195 handle_command_reply: tid=0 2024-04-17T07:57:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 client.25195 shutdown 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 msgr2=0x556ad121d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad121d360 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0278006010 tx=0x7f0278009680 comp rx=0 tx=0).stop 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 shutdown_connections 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1229e10 0x556ad122b280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad1218440 0x556ad12243a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12132d0 0x556ad121d360 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 --2- 172.21.15.46:0/4000471716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad121ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f0274c00640 0 client.25195 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 >> 172.21.15.46:0/4000471716 conn(0x556ad120de00 msgr2=0x556ad120d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 shutdown_connections 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 1 -- 172.21.15.46:0/4000471716 wait complete. 2024-04-17T07:57:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.904+0000 7f02945e1740 20 client.25195 trim_cache size 0 max 16384 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.924+0000 7f02945e1740 1 Processor -- start 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.924+0000 7f02945e1740 1 -- start start 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f02945e1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12128e0 0x556ad120c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f02945e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad1224360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f02945e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad1218440 0x556ad120cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f02945e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ad12295c0 con 0x556ad12132d0 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f02945e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556ad1229c90 con 0x556ad1218440 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f02945e1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ad122a360 con 0x556ad12128e0 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad1224360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12128e0 0x556ad120c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad1224360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38504/0 (socket says 172.21.15.46:38504) 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12128e0 0x556ad120c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:33016/0 (socket says 172.21.15.46:33016) 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 -- 172.21.15.46:0/1890941587 learned_addr learned my addr 172.21.15.46:0/1890941587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028b400640 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad1218440 0x556ad120cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 -- 172.21.15.46:0/1890941587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12128e0 msgr2=0x556ad120c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12128e0 0x556ad120c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 -- 172.21.15.46:0/1890941587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad1218440 msgr2=0x556ad120cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad1218440 0x556ad120cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.925+0000 7f028a000640 1 -- 172.21.15.46:0/1890941587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad120ebc0 con 0x556ad12132d0 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.926+0000 7f028a000640 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad1224360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0278003b80 tx=0x7f0278009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c305d13459dfbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.926+0000 7f0277400640 1 -- 172.21.15.46:0/1890941587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278000e60 con 0x556ad12132d0 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.926+0000 7f0277400640 1 -- 172.21.15.46:0/1890941587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02780035f0 con 0x556ad12132d0 2024-04-17T07:57:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.926+0000 7f0277400640 1 -- 172.21.15.46:0/1890941587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278004900 con 0x556ad12132d0 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 -- 172.21.15.46:0/1890941587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 msgr2=0x556ad1224360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad1224360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0278003b80 tx=0x7f0278009520 comp rx=0 tx=0).stop 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 -- 172.21.15.46:0/1890941587 shutdown_connections 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad1218440 0x556ad120cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad1224360 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 --2- 172.21.15.46:0/1890941587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad12128e0 0x556ad120c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 -- 172.21.15.46:0/1890941587 >> 172.21.15.46:0/1890941587 conn(0x556ad120de00 msgr2=0x556ad120e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 -- 172.21.15.46:0/1890941587 shutdown_connections 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 -- 172.21.15.46:0/1890941587 wait complete. 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.927+0000 7f02945e1740 1 -- start start 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 1 Processor -- start 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1220060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad12205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad1218440 0x556ad12235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ad122b100 con 0x556ad12132d0 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556ad122aa30 con 0x556ad12128e0 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556ad122b7d0 con 0x556ad1218440 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f02945e1740 10 client.0 _ll_register_callbacks cb 0x7f0278009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f028a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad12205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f028b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1220060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f028a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad12205a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38512/0 (socket says 172.21.15.46:38512) 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.928+0000 7f028a000640 1 -- 172.21.15.46:0/1188274494 learned_addr learned my addr 172.21.15.46:0/1188274494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028b400640 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad1218440 0x556ad12235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028a000640 1 -- 172.21.15.46:0/1188274494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad1218440 msgr2=0x556ad12235c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028a000640 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad1218440 0x556ad12235c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028a000640 1 -- 172.21.15.46:0/1188274494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 msgr2=0x556ad1220060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028a000640 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1220060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028a000640 1 -- 172.21.15.46:0/1188274494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad1223bb0 con 0x556ad12132d0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028b400640 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1220060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f028a000640 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad12205a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0278009850 tx=0x7f0278004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2f2f95491fcf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f0274c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f0274c00640 1 -- 172.21.15.46:0/1188274494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278042020 con 0x556ad12132d0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f0274c00640 1 -- 172.21.15.46:0/1188274494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f02780052e0 con 0x556ad12132d0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.929+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02780035f0 con 0x556ad12132d0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f0274c00640 1 -- 172.21.15.46:0/1188274494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0278005d50 con 0x556ad12132d0 2024-04-17T07:57:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 10 client.15084 fetch_fsmap learned FSMap version 23 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556ad12903e0 con 0x556ad12132d0 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f0274c00640 1 -- 172.21.15.46:0/1188274494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02780035f0 con 0x556ad12132d0 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f0274c00640 1 -- 172.21.15.46:0/1188274494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7f0278054070 con 0x556ad12132d0 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 10 client.15084 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 20 client.15084 populate_metadata read hostname 'smithi046' 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1294da0 0x556ad1293ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 4 client.15084 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.930+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "status"}) -- 0x556ad1181210 con 0x556ad1294da0 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.931+0000 7f028b400640 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1294da0 0x556ad1293ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.931+0000 7f028b400640 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1294da0 0x556ad1293ab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96e771238dce4518 server_cookie=9cda35f280b11c08 in_seq=0 out_seq=0 2024-04-17T07:57:54.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.931+0000 7f0274c00640 10 client.15084 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:57:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.931+0000 7f0274c00640 1 -- 172.21.15.46:0/1188274494 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556ad1181210 con 0x556ad1294da0 2024-04-17T07:57:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.931+0000 7f0274c00640 10 client.15084 handle_command_reply: tid=0 2024-04-17T07:57:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.931+0000 7f02945e1740 1 client.15084 shutdown 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 msgr2=0x556ad12205a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad12205a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0278009850 tx=0x7f0278004370 comp rx=0 tx=0).stop 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 shutdown_connections 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556ad1218440 0x556ad12235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x556ad1294da0 0x556ad1293ab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ad12132d0 0x556ad12205a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 --2- 172.21.15.46:0/1188274494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556ad12128e0 0x556ad1220060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f0274c00640 0 client.15084 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 >> 172.21.15.46:0/1188274494 conn(0x556ad120de00 msgr2=0x556ad12110b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 shutdown_connections 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 1 -- 172.21.15.46:0/1188274494 wait complete. 2024-04-17T07:57:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.932+0000 7f02945e1740 20 client.15084 trim_cache size 0 max 16384 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24556, 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 122.225948227, 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 23, 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 162.28047776299999 2024-04-17T07:57:54.936 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:57:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f02700a3bb0 msgr2=0x7f02700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f02700a3bb0 0x7f02700a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f027c098e30 tx=0x7f027c00efb0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 msgr2=0x7f028c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 0x7f028c19a500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9c0 tx=0x7f027c00ee80 comp rx=0 tx=0).stop 2024-04-17T07:57:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 shutdown_connections 2024-04-17T07:57:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f02700a3bb0 0x7f02700a6070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f028c105d60 0x7f028c1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f028c1053a0 0x7f028c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 --2- 172.21.15.46:0/1783369397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c1041b0 0x7f028c19a500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 >> 172.21.15.46:0/1783369397 conn(0x7f028c0ff720 msgr2=0x7f028c108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 shutdown_connections 2024-04-17T07:57:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:54.933+0000 7f0291800640 1 -- 172.21.15.46:0/1783369397 wait complete. 2024-04-17T07:57:54.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T07:57:55.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 -- 172.21.15.46:0/1840556514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 msgr2=0x7ffb7c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 --2- 172.21.15.46:0/1840556514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c100cb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009860 tx=0x7ffb6c02bc10 comp rx=0 tx=0).stop 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 -- 172.21.15.46:0/1840556514 shutdown_connections 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 --2- 172.21.15.46:0/1840556514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb7c101b90 0x7ffb7c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 --2- 172.21.15.46:0/1840556514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb7c1011f0 0x7ffb7c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 --2- 172.21.15.46:0/1840556514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c100cb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 -- 172.21.15.46:0/1840556514 >> 172.21.15.46:0/1840556514 conn(0x7ffb7c0fc430 msgr2=0x7ffb7c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 -- 172.21.15.46:0/1840556514 shutdown_connections 2024-04-17T07:57:55.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.163+0000 7ffb83800640 1 -- 172.21.15.46:0/1840556514 wait complete. 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.164+0000 7ffb83800640 1 Processor -- start 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.164+0000 7ffb83800640 1 -- start start 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb83800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb83800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb7c1011f0 0x7ffb7c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb83800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb7c101b90 0x7ffb7c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb83800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10fe30 con 0x7ffb7c1011f0 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb83800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10fcb0 con 0x7ffb7c100890 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb83800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10ffb0 con 0x7ffb7c101b90 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb7c1011f0 0x7ffb7c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb80e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb80e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51696/0 (socket says 172.21.15.46:51696) 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb7c1011f0 0x7ffb7c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38532/0 (socket says 172.21.15.46:38532) 2024-04-17T07:57:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb80e00640 1 -- 172.21.15.46:0/1003902745 learned_addr learned my addr 172.21.15.46:0/1003902745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.165+0000 7ffb81800640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb7c101b90 0x7ffb7c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb80e00640 1 -- 172.21.15.46:0/1003902745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb7c101b90 msgr2=0x7ffb7c1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb80e00640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb7c101b90 0x7ffb7c1a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb80e00640 1 -- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb7c1011f0 msgr2=0x7ffb7c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb80e00640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb7c1011f0 0x7ffb7c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb80e00640 1 -- 172.21.15.46:0/1003902745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c1a20e0 con 0x7ffb7c100890 2024-04-17T07:57:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb80e00640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c19a4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005940 tx=0x7ffb6c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb7a000640 1 -- 172.21.15.46:0/1003902745 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c041020 con 0x7ffb7c100890 2024-04-17T07:57:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c1a2370 con 0x7ffb7c100890 2024-04-17T07:57:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb7a000640 1 -- 172.21.15.46:0/1003902745 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c0035f0 con 0x7ffb7c100890 2024-04-17T07:57:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb7a000640 1 -- 172.21.15.46:0/1003902745 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c039040 con 0x7ffb7c100890 2024-04-17T07:57:55.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.166+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c1a28b0 con 0x7ffb7c100890 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.171+0000 7ffb86407740 1 Processor -- start 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.171+0000 7ffb86407740 1 -- start start 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.171+0000 7ffb7a000640 1 -- 172.21.15.46:0/1003902745 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffb6c044020 con 0x7ffb7c100890 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.171+0000 7ffb86407740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c0f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb86407740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f32d0 0x55f41c0f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb86407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c0fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb86407740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f41c059980 con 0x55f41c0f28e0 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb86407740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f41c000220 con 0x55f41c0f32d0 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb86407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f41c055940 con 0x55f41c0f8440 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c0f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb80e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f32d0 0x55f41c0f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c0fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c0f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38534/0 (socket says 172.21.15.46:38534) 2024-04-17T07:57:55.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 -- 172.21.15.46:0/442002721 learned_addr learned my addr 172.21.15.46:0/442002721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 -- 172.21.15.46:0/442002721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 msgr2=0x55f41c0fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 --2- 172.21.15.46:0/442002721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c0fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 -- 172.21.15.46:0/442002721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f32d0 msgr2=0x55f41c0f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 --2- 172.21.15.46:0/442002721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f32d0 0x55f41c0f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.172+0000 7ffb81800640 1 -- 172.21.15.46:0/442002721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f41bd6c2d0 con 0x55f41c0f28e0 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.173+0000 7ffb81800640 1 --2- 172.21.15.46:0/442002721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c0f2d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009fd0 tx=0x7ffb740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634ecbf68727c6dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.173+0000 7ffb7a000640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffb600a3bb0 0x7ffb600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.173+0000 7ffb72000640 1 -- 172.21.15.46:0/442002721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74004d20 con 0x55f41c0f28e0 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.173+0000 7ffb72000640 1 -- 172.21.15.46:0/442002721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb74004050 con 0x55f41c0f28e0 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.173+0000 7ffb72000640 1 -- 172.21.15.46:0/442002721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74005280 con 0x55f41c0f28e0 2024-04-17T07:57:55.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.173+0000 7ffb7a000640 1 -- 172.21.15.46:0/1003902745 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ffb6c0f20c0 con 0x7ffb7c100890 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.173+0000 7ffb80e00640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffb600a3bb0 0x7ffb600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 -- 172.21.15.46:0/442002721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 msgr2=0x55f41c0f2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 --2- 172.21.15.46:0/442002721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c0f2d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009fd0 tx=0x7ffb740118d0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 -- 172.21.15.46:0/442002721 shutdown_connections 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 --2- 172.21.15.46:0/442002721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c0fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 --2- 172.21.15.46:0/442002721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f32d0 0x55f41c0f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 --2- 172.21.15.46:0/442002721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c0f2d00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 -- 172.21.15.46:0/442002721 >> 172.21.15.46:0/442002721 conn(0x55f41c0ede00 msgr2=0x55f41c0ee200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 -- 172.21.15.46:0/442002721 shutdown_connections 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb80e00640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffb600a3bb0 0x7ffb600a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009830 tx=0x7ffb6c0f5b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.174+0000 7ffb86407740 1 -- 172.21.15.46:0/442002721 wait complete. 2024-04-17T07:57:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 -- start start 2024-04-17T07:57:55.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 Processor -- start 2024-04-17T07:57:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f32d0 0x55f41c0fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c104380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f41c0fb3f0 con 0x55f41c0f32d0 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f41c013590 con 0x55f41c0f28e0 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f41c0fb570 con 0x55f41c0f8440 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.175+0000 7ffb86407740 10 client.0 _ll_register_callbacks cb 0x55f41bd6bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f32d0 0x55f41c0fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f32d0 0x55f41c0fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38542/0 (socket says 172.21.15.46:38542) 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 -- 172.21.15.46:0/2411729638 learned_addr learned my addr 172.21.15.46:0/2411729638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb81800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb81800640 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 -- 172.21.15.46:0/2411729638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 msgr2=0x55f41c104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 -- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 msgr2=0x55f41c0fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 -- 172.21.15.46:0/2411729638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f41be366b0 con 0x55f41c0f32d0 2024-04-17T07:57:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb73e00640 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f32d0 0x55f41c0fd340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffb640116d0 tx=0x7ffb64011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22f3f856f2d152d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.176+0000 7ffb6aa00640 1 -- 172.21.15.46:0/2411729638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6401e060 con 0x55f41c0f32d0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.177+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb74004050 con 0x55f41c0f32d0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.177+0000 7ffb6aa00640 1 -- 172.21.15.46:0/2411729638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ffb6400fb90 con 0x55f41c0f32d0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.177+0000 7ffb6aa00640 1 -- 172.21.15.46:0/2411729638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffb64025050 con 0x55f41c0f32d0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.177+0000 7ffb86407740 10 client.15096 fetch_fsmap learned FSMap version 23 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.177+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f41c1077c0 con 0x55f41c0f32d0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.177+0000 7ffb6aa00640 1 -- 172.21.15.46:0/2411729638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb74004050 con 0x55f41c0f32d0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.177+0000 7ffb6aa00640 1 -- 172.21.15.46:0/2411729638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7ffb64019070 con 0x55f41c0f32d0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.178+0000 7ffb86407740 10 client.15096 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.178+0000 7ffb86407740 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.178+0000 7ffb86407740 20 client.15096 populate_metadata read hostname 'smithi046' 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.178+0000 7ffb86407740 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c109eb0 0x55f41c10b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.178+0000 7ffb86407740 4 client.15096 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.178+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f41c0673a0 con 0x55f41c109eb0 2024-04-17T07:57:55.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.178+0000 7ffb81800640 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c109eb0 0x55f41c10b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.179+0000 7ffb81800640 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c109eb0 0x55f41c10b320 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25640256fc1bdbcd server_cookie=c6ec36d1e0586997 in_seq=0 out_seq=0 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.179+0000 7ffb6aa00640 10 client.15096 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.179+0000 7ffb6aa00640 1 -- 172.21.15.46:0/2411729638 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f41c0673a0 con 0x55f41c109eb0 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.179+0000 7ffb6aa00640 10 client.15096 handle_command_reply: tid=0 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 client.15096 shutdown 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f32d0 msgr2=0x55f41c0fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f32d0 0x55f41c0fd340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffb640116d0 tx=0x7ffb64011b90 comp rx=0 tx=0).stop 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 shutdown_connections 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c109eb0 0x55f41c10b320 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f32d0 0x55f41c0fd340 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 --2- 172.21.15.46:0/2411729638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb6aa00640 0 client.15096 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:57:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 >> 172.21.15.46:0/2411729638 conn(0x55f41c0ede00 msgr2=0x55f41c0ed090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 shutdown_connections 2024-04-17T07:57:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 1 -- 172.21.15.46:0/2411729638 wait complete. 2024-04-17T07:57:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.180+0000 7ffb86407740 20 client.15096 trim_cache size 0 max 16384 2024-04-17T07:57:55.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.200+0000 7ffb86407740 1 Processor -- start 2024-04-17T07:57:55.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.200+0000 7ffb86407740 1 -- start start 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb86407740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb86407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f32d0 0x55f41c104340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb86407740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f8440 0x55f41c0ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb86407740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f41c109660 con 0x55f41c0f8440 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb86407740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f41c109d30 con 0x55f41c0f28e0 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb86407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f41c10a400 con 0x55f41c0f32d0 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f32d0 0x55f41c104340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f32d0 0x55f41c104340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:33058/0 (socket says 172.21.15.46:33058) 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 -- 172.21.15.46:0/3462275421 learned_addr learned my addr 172.21.15.46:0/3462275421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb81800640 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb81800640 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f8440 0x55f41c0ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 -- 172.21.15.46:0/3462275421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 msgr2=0x55f41c0fb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 -- 172.21.15.46:0/3462275421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f8440 msgr2=0x55f41c0ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f8440 0x55f41c0ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.201+0000 7ffb73e00640 1 -- 172.21.15.46:0/3462275421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f41c0f3b60 con 0x55f41c0f32d0 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.202+0000 7ffb73e00640 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f32d0 0x55f41c104340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffb64003da0 tx=0x7ffb64009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b1418a27c83b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.202+0000 7ffb71600640 1 -- 172.21.15.46:0/3462275421 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6402d020 con 0x55f41c0f32d0 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.202+0000 7ffb71600640 1 -- 172.21.15.46:0/3462275421 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6400fb90 con 0x55f41c0f32d0 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.202+0000 7ffb71600640 1 -- 172.21.15.46:0/3462275421 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6401a440 con 0x55f41c0f32d0 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 -- 172.21.15.46:0/3462275421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f32d0 msgr2=0x55f41c104340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f32d0 0x55f41c104340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffb64003da0 tx=0x7ffb64009f00 comp rx=0 tx=0).stop 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 -- 172.21.15.46:0/3462275421 shutdown_connections 2024-04-17T07:57:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f8440 0x55f41c0ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f32d0 0x55f41c104340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 --2- 172.21.15.46:0/3462275421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c0f28e0 0x55f41c0fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 -- 172.21.15.46:0/3462275421 >> 172.21.15.46:0/3462275421 conn(0x55f41c0ede00 msgr2=0x55f41c0e90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 -- 172.21.15.46:0/3462275421 shutdown_connections 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 -- 172.21.15.46:0/3462275421 wait complete. 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.203+0000 7ffb86407740 1 -- start start 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 1 Processor -- start 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c1004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c100a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c1033d0 0x55f41c103850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f41c10b1a0 con 0x55f41c0f28e0 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f41c10aad0 con 0x55f41c1033d0 2024-04-17T07:57:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f41c10b870 con 0x55f41c0f8440 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb86407740 10 client.0 _ll_register_callbacks cb 0x7ffb64009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb81800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c1004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb81800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c1004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38556/0 (socket says 172.21.15.46:38556) 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.204+0000 7ffb81800640 1 -- 172.21.15.46:0/161396689 learned_addr learned my addr 172.21.15.46:0/161396689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb73e00640 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb81800640 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c1033d0 0x55f41c103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb81800640 1 -- 172.21.15.46:0/161396689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 msgr2=0x55f41c100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb81800640 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb81800640 1 -- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c1033d0 msgr2=0x55f41c103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb81800640 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c1033d0 0x55f41c103850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb81800640 1 -- 172.21.15.46:0/161396689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f41c103dc0 con 0x55f41c0f28e0 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb81800640 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c1004d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb74004cc0 tx=0x7ffb740039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9660a5d76ccf9c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:57:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb6aa00640 1 -- 172.21.15.46:0/161396689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7402e0c0 con 0x55f41c0f28e0 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb6aa00640 1 -- 172.21.15.46:0/161396689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ffb7402e260 con 0x55f41c0f28e0 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.205+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb6400fb90 con 0x55f41c0f28e0 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb6aa00640 1 -- 172.21.15.46:0/161396689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffb7402e460 con 0x55f41c0f28e0 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 10 client.15102 fetch_fsmap learned FSMap version 23 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f41c170710 con 0x55f41c0f28e0 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb6aa00640 1 -- 172.21.15.46:0/161396689 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6400fb90 con 0x55f41c0f28e0 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb6aa00640 1 -- 172.21.15.46:0/161396689 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7ffb74037070 con 0x55f41c0f28e0 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 10 client.15102 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 20 client.15102 populate_metadata read hostname 'smithi046' 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c1751b0 0x55f41c173ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 4 client.15102 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:57:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.206+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "status"}) -- 0x55f41c061210 con 0x55f41c1751b0 2024-04-17T07:57:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.207+0000 7ffb73e00640 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c1751b0 0x55f41c173ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.207+0000 7ffb73e00640 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c1751b0 0x55f41c173ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c3cea99b5d4435 server_cookie=8cb6b5452b9510d3 in_seq=0 out_seq=0 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.207+0000 7ffb6aa00640 10 client.15102 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb6aa00640 1 -- 172.21.15.46:0/161396689 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f41c061210 con 0x55f41c1751b0 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb6aa00640 10 client.15102 handle_command_reply: tid=0 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 client.15102 shutdown 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 msgr2=0x55f41c1004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c1004d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb74004cc0 tx=0x7ffb740039b0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 shutdown_connections 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55f41c1751b0 0x55f41c173ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f41c1033d0 0x55f41c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f41c0f8440 0x55f41c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 --2- 172.21.15.46:0/161396689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f41c0f28e0 0x55f41c1004d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 >> 172.21.15.46:0/161396689 conn(0x55f41c0ede00 msgr2=0x55f41c0ee1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb6aa00640 0 client.15102 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 shutdown_connections 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 1 -- 172.21.15.46:0/161396689 wait complete. 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.208+0000 7ffb86407740 20 client.15102 trim_cache size 0 max 16384 2024-04-17T07:57:55.212 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24596, 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 148.66579428099999, 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 23, 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 155.16969797100001 2024-04-17T07:57:55.213 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:57:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffb600a3bb0 msgr2=0x7ffb600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffb600a3bb0 0x7ffb600a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009830 tx=0x7ffb6c0f5b00 comp rx=0 tx=0).stop 2024-04-17T07:57:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 msgr2=0x7ffb7c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c19a4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005940 tx=0x7ffb6c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 shutdown_connections 2024-04-17T07:57:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffb600a3bb0 0x7ffb600a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb7c101b90 0x7ffb7c1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb7c1011f0 0x7ffb7c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 --2- 172.21.15.46:0/1003902745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb7c100890 0x7ffb7c19a4c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 >> 172.21.15.46:0/1003902745 conn(0x7ffb7c0fc430 msgr2=0x7ffb7c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 shutdown_connections 2024-04-17T07:57:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:55.209+0000 7ffb83800640 1 -- 172.21.15.46:0/1003902745 wait complete. 2024-04-17T07:57:55.227 INFO:tasks.cephadm:Setup complete, yielding 2024-04-17T07:57:55.227 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-17T07:57:55.237 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-04-17T07:57:55.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch status' 2024-04-17T07:57:55.322 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:55 smithi046 ceph-mon[32038]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:55.322 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:55 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4084040478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:55.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:55 smithi053 ceph-mon[36437]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:55.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:55 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4084040478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:55.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:55 smithi062 ceph-mon[36507]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:57:55.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:55 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4084040478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:57:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:57:57.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:57 smithi046 ceph-mon[32038]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:57.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:57 smithi053 ceph-mon[36437]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:57.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:57 smithi062 ceph-mon[36507]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:57:57.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 -- 172.21.15.46:0/305617865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 msgr2=0x7f76ec109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:57.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 --2- 172.21.15.46:0/305617865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 0x7f76ec109510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009990 tx=0x7f76dc009ca0 comp rx=0 tx=0).stop 2024-04-17T07:57:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 -- 172.21.15.46:0/305617865 shutdown_connections 2024-04-17T07:57:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 --2- 172.21.15.46:0/305617865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10ac90 0x7f76ec111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 --2- 172.21.15.46:0/305617865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 0x7f76ec109510 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 --2- 172.21.15.46:0/305617865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76ec10a2f0 0x7f76ec10a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 -- 172.21.15.46:0/305617865 >> 172.21.15.46:0/305617865 conn(0x7f76ec0780c0 msgr2=0x7f76ec07a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 -- 172.21.15.46:0/305617865 shutdown_connections 2024-04-17T07:57:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.985+0000 7f76f1a00700 1 -- 172.21.15.46:0/305617865 wait complete. 2024-04-17T07:57:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.986+0000 7f76f1a00700 1 Processor -- start 2024-04-17T07:57:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.986+0000 7f76f1a00700 1 -- start start 2024-04-17T07:57:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76f1a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 0x7f76ec19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76f1a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10a2f0 0x7f76ec19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76f1a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76ec10ac90 0x7f76ec1a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76f1a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec114d30 con 0x7f76ec10ac90 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76f1a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec114bb0 con 0x7f76ec10a2f0 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76f1a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec114eb0 con 0x7f76ec1090f0 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76ea000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10a2f0 0x7f76ec19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76eb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76ec10ac90 0x7f76ec1a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76ea000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10a2f0 0x7f76ec19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51614/0 (socket says 172.21.15.46:51614) 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76ea000700 1 -- 172.21.15.46:0/3427980583 learned_addr learned my addr 172.21.15.46:0/3427980583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.987+0000 7f76eaa00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 0x7f76ec19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76ea000700 1 -- 172.21.15.46:0/3427980583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 msgr2=0x7f76ec19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76ea000700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 0x7f76ec19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76ea000700 1 -- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76ec10ac90 msgr2=0x7f76ec1a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76ea000700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76ec10ac90 0x7f76ec1a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76ea000700 1 -- 172.21.15.46:0/3427980583 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec1a6f70 con 0x7f76ec10a2f0 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76ea000700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10a2f0 0x7f76ec19f850 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76d800e9a0 tx=0x7f76d800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76e3400700 1 -- 172.21.15.46:0/3427980583 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d800ca10 con 0x7f76ec10a2f0 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ec1a7260 con 0x7f76ec10a2f0 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76e3400700 1 -- 172.21.15.46:0/3427980583 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d800cbb0 con 0x7f76ec10a2f0 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.988+0000 7f76e3400700 1 -- 172.21.15.46:0/3427980583 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8018710 con 0x7f76ec10a2f0 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.989+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ec1a77f0 con 0x7f76ec10a2f0 2024-04-17T07:57:57.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.990+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ec069740 con 0x7f76ec10a2f0 2024-04-17T07:57:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.992+0000 7f76e3400700 1 -- 172.21.15.46:0/3427980583 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f76d80188b0 con 0x7f76ec10a2f0 2024-04-17T07:57:57.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.993+0000 7f76e3400700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76d00a3c40 0x7f76d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:57:57.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.993+0000 7f76e3400700 1 -- 172.21.15.46:0/3427980583 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f76d80d0b80 con 0x7f76ec10a2f0 2024-04-17T07:57:57.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.993+0000 7f76eaa00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76d00a3c40 0x7f76d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:57:57.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.994+0000 7f76eaa00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76d00a3c40 0x7f76d00a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76dc000c00 tx=0x7f76dc00b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:57:57.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:57.995+0000 7f76e3400700 1 -- 172.21.15.46:0/3427980583 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f76d809d170 con 0x7f76ec10a2f0 2024-04-17T07:57:58.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.251+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f76ec062f50 con 0x7f76d00a3c40 2024-04-17T07:57:58.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.253+0000 7f76e3400700 1 -- 172.21.15.46:0/3427980583 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f76ec062f50 con 0x7f76d00a3c40 2024-04-17T07:57:58.253 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-04-17T07:57:58.253 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-04-17T07:57:58.253 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-04-17T07:57:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76d00a3c40 msgr2=0x7f76d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76d00a3c40 0x7f76d00a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76dc000c00 tx=0x7f76dc00b8f0 comp rx=0 tx=0).stop 2024-04-17T07:57:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10a2f0 msgr2=0x7f76ec19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:57:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10a2f0 0x7f76ec19f850 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76d800e9a0 tx=0x7f76d800ecb0 comp rx=0 tx=0).stop 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 shutdown_connections 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76d00a3c40 0x7f76d00a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76ec10ac90 0x7f76ec1a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76ec1090f0 0x7f76ec19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 --2- 172.21.15.46:0/3427980583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76ec10a2f0 0x7f76ec19f850 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 >> 172.21.15.46:0/3427980583 conn(0x7f76ec0780c0 msgr2=0x7f76ec10d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 shutdown_connections 2024-04-17T07:57:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:57:58.257+0000 7f76f1a00700 1 -- 172.21.15.46:0/3427980583 wait complete. 2024-04-17T07:57:59.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-17T07:57:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:59 smithi053 ceph-mon[36437]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:57:59 smithi053 ceph-mon[36437]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:57:59.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:59 smithi062 ceph-mon[36507]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:57:59 smithi062 ceph-mon[36507]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:57:59.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:59 smithi046 ceph-mon[32038]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:57:59.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:57:59 smithi046 ceph-mon[32038]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:57:59.672 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:58:01.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:01 smithi046 ceph-mon[32038]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:01.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:01 smithi053 ceph-mon[36437]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:01.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:01 smithi062 ceph-mon[36507]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 -- 172.21.15.46:0/387491815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c103470 msgr2=0x7f8e8c105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 --2- 172.21.15.46:0/387491815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c103470 0x7f8e8c105860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0098f0 tx=0x7f8e7c009c00 comp rx=0 tx=0).stop 2024-04-17T07:58:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 -- 172.21.15.46:0/387491815 shutdown_connections 2024-04-17T07:58:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 --2- 172.21.15.46:0/387491815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c1086d0 0x7f8e8c10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 --2- 172.21.15.46:0/387491815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c103470 0x7f8e8c105860 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 --2- 172.21.15.46:0/387491815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c105da0 0x7f8e8c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 -- 172.21.15.46:0/387491815 >> 172.21.15.46:0/387491815 conn(0x7f8e8c0fce60 msgr2=0x7f8e8c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.066+0000 7f8e91800700 1 -- 172.21.15.46:0/387491815 shutdown_connections 2024-04-17T07:58:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.067+0000 7f8e91800700 1 -- 172.21.15.46:0/387491815 wait complete. 2024-04-17T07:58:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.068+0000 7f8e91800700 1 Processor -- start 2024-04-17T07:58:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.068+0000 7f8e91800700 1 -- start start 2024-04-17T07:58:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.068+0000 7f8e91800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c103470 0x7f8e8c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e91800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c105da0 0x7f8e8c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e91800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c1086d0 0x7f8e8c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e91800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10e5b0 con 0x7f8e8c105da0 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e91800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10e430 con 0x7f8e8c1086d0 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e91800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10e730 con 0x7f8e8c103470 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e83400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c105da0 0x7f8e8c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c103470 0x7f8e8c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c103470 0x7f8e8c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50130/0 (socket says 172.21.15.46:50130) 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 -- 172.21.15.46:0/402398241 learned_addr learned my addr 172.21.15.46:0/402398241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e83400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c105da0 0x7f8e8c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44336/0 (socket says 172.21.15.46:44336) 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e90e00700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c1086d0 0x7f8e8c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 -- 172.21.15.46:0/402398241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c1086d0 msgr2=0x7f8e8c1a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c1086d0 0x7f8e8c1a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 -- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c105da0 msgr2=0x7f8e8c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c105da0 0x7f8e8c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.069+0000 7f8e8be00700 1 -- 172.21.15.46:0/402398241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c1a2a00 con 0x7f8e8c103470 2024-04-17T07:58:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.070+0000 7f8e8be00700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c103470 0x7f8e8c19adb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009810 tx=0x7f8e7c005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.070+0000 7f8e89600700 1 -- 172.21.15.46:0/402398241 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0054f0 con 0x7f8e8c103470 2024-04-17T07:58:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.070+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1a2c90 con 0x7f8e8c103470 2024-04-17T07:58:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.071+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a3140 con 0x7f8e8c103470 2024-04-17T07:58:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.071+0000 7f8e89600700 1 -- 172.21.15.46:0/402398241 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7c005690 con 0x7f8e8c103470 2024-04-17T07:58:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.071+0000 7f8e89600700 1 -- 172.21.15.46:0/402398241 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c003d00 con 0x7f8e8c103470 2024-04-17T07:58:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.072+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8c069740 con 0x7f8e8c103470 2024-04-17T07:58:02.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.074+0000 7f8e89600700 1 -- 172.21.15.46:0/402398241 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8e7c00b560 con 0x7f8e8c103470 2024-04-17T07:58:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.075+0000 7f8e89600700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8e700a3cc0 0x7f8e700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.075+0000 7f8e89600700 1 -- 172.21.15.46:0/402398241 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8e7c0d17c0 con 0x7f8e8c103470 2024-04-17T07:58:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.075+0000 7f8e83400700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8e700a3cc0 0x7f8e700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.076+0000 7f8e83400700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8e700a3cc0 0x7f8e700a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009c60 tx=0x7f8e74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.080+0000 7f8e89600700 1 -- 172.21.15.46:0/402398241 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8e7c013030 con 0x7f8e8c103470 2024-04-17T07:58:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.323+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8e8c062f50 con 0x7f8e700a3cc0 2024-04-17T07:58:02.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.363+0000 7f8e89600700 1 -- 172.21.15.46:0/402398241 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f8e8c062f50 con 0x7f8e700a3cc0 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.a smithi046 running (2m) 48s ago 2m 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 bbe75030bc0a 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.b smithi053 running (2m) 93s ago 2m 26.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 98169a4dcdb2 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.c smithi062 running (3m) 93s ago 3m 16.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 0d86076c3e17 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.d smithi046 running (3m) 48s ago 3m 12.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 af92cf60c6f9 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.e smithi053 running (3m) 93s ago 3m 15.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 ed7c78c5c86c 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.f smithi062 running (3m) 93s ago 3m 16.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 ab2a5170b6c8 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.g smithi046 running (2m) 48s ago 2m 16.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 b88ece623d38 2024-04-17T07:58:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:mds.h smithi053 running (2m) 93s ago 2m 18.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 cc51d3d83da7 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mds.i smithi062 running (2m) 93s ago 2m 16.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 d3bbd843e1eb 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mds.j smithi046 running (3m) 48s ago 3m 12.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 65861081c639 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mds.k smithi053 running (3m) 93s ago 3m 16.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 236db5994509 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mds.l smithi062 running (3m) 93s ago 3m 16.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 a56f0800ff73 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.x smithi046 *:9283,8765 running (11m) 48s ago 11m 516M - 19.0.0-3133-ga8c84f59 76bc6b91b726 4e3930768b4a 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.y smithi053 *:8443,8765 running (9m) 93s ago 9m 451M - 19.0.0-3133-ga8c84f59 76bc6b91b726 10ba0a837158 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.z smithi062 *:8443,8765 running (9m) 93s ago 9m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 35a0481921de 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a smithi046 running (11m) 48s ago 11m 48.5M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 18918c5ec8f3 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b smithi053 running (9m) 93s ago 9m 40.5M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 4941649dc87c 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c smithi062 running (9m) 93s ago 9m 44.2M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 f7dc232f5d83 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 48s ago 8m 45.1M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 38ac4eb509b9 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi053 running (8m) 93s ago 8m 47.2M 4271M 19.0.0-3133-ga8c84f59 76bc6b91b726 7b3e8fd1269b 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi062 running (8m) 93s ago 8m 55.3M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 1122ab56a221 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi046 running (7m) 48s ago 7m 52.0M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 5c754b60b7f9 2024-04-17T07:58:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi053 running (7m) 93s ago 7m 49.2M 4271M 19.0.0-3133-ga8c84f59 76bc6b91b726 0c47e08fe141 2024-04-17T07:58:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi062 running (6m) 93s ago 6m 53.7M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 05d95c3be5f6 2024-04-17T07:58:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 smithi046 running (6m) 48s ago 6m 41.8M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 a9095f342142 2024-04-17T07:58:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 smithi053 running (5m) 93s ago 5m 45.1M 4271M 19.0.0-3133-ga8c84f59 76bc6b91b726 a57362dae851 2024-04-17T07:58:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 smithi062 running (5m) 93s ago 5m 51.1M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 4592ca009bfa 2024-04-17T07:58:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 smithi046 running (4m) 48s ago 4m 42.0M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 aa0206653c5d 2024-04-17T07:58:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 smithi053 running (4m) 93s ago 4m 42.9M 4271M 19.0.0-3133-ga8c84f59 76bc6b91b726 d0d883f9eb47 2024-04-17T07:58:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 smithi062 running (3m) 93s ago 3m 50.2M 4270M 19.0.0-3133-ga8c84f59 76bc6b91b726 29002b1f2e1c 2024-04-17T07:58:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8e700a3cc0 msgr2=0x7f8e700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8e700a3cc0 0x7f8e700a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009c60 tx=0x7f8e74009260 comp rx=0 tx=0).stop 2024-04-17T07:58:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c103470 msgr2=0x7f8e8c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c103470 0x7f8e8c19adb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009810 tx=0x7f8e7c005250 comp rx=0 tx=0).stop 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 shutdown_connections 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8e700a3cc0 0x7f8e700a6180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c1086d0 0x7f8e8c1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e8c103470 0x7f8e8c19adb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.367+0000 7f8e91800700 1 --2- 172.21.15.46:0/402398241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c105da0 0x7f8e8c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.368+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 >> 172.21.15.46:0/402398241 conn(0x7f8e8c0fce60 msgr2=0x7f8e8c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.368+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 shutdown_connections 2024-04-17T07:58:02.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:02.368+0000 7f8e91800700 1 -- 172.21.15.46:0/402398241 wait complete. 2024-04-17T07:58:03.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-17T07:58:03.440 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:03 smithi046 ceph-mon[32038]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:03 smithi053 ceph-mon[36437]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:03.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:03 smithi062 ceph-mon[36507]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:03.789 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:58:04.290 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:04 smithi046 ceph-mon[32038]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:04.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:04 smithi053 ceph-mon[36437]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:04.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:04 smithi062 ceph-mon[36507]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:05.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:05 smithi046 ceph-mon[32038]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:05.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:05 smithi053 ceph-mon[36437]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:05.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:05 smithi062 ceph-mon[36507]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.927+0000 7f76d2000700 1 -- 172.21.15.46:0/3047107214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 msgr2=0x7f76cc109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.927+0000 7f76d2000700 1 --2- 172.21.15.46:0/3047107214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc109510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009990 tx=0x7f76bc009ca0 comp rx=0 tx=0).stop 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.928+0000 7f76d2000700 1 -- 172.21.15.46:0/3047107214 shutdown_connections 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.928+0000 7f76d2000700 1 --2- 172.21.15.46:0/3047107214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76cc10ac90 0x7f76cc111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.928+0000 7f76d2000700 1 --2- 172.21.15.46:0/3047107214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc109510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.928+0000 7f76d2000700 1 --2- 172.21.15.46:0/3047107214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76cc10a2f0 0x7f76cc10a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.928+0000 7f76d2000700 1 -- 172.21.15.46:0/3047107214 >> 172.21.15.46:0/3047107214 conn(0x7f76cc0780c0 msgr2=0x7f76cc07a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.928+0000 7f76d2000700 1 -- 172.21.15.46:0/3047107214 shutdown_connections 2024-04-17T07:58:05.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.928+0000 7f76d2000700 1 -- 172.21.15.46:0/3047107214 wait complete. 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 Processor -- start 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 -- start start 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76cc10a2f0 0x7f76cc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76cc10ac90 0x7f76cc1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc114d00 con 0x7f76cc10ac90 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc114b80 con 0x7f76cc10a2f0 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.929+0000 7f76d2000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc114e80 con 0x7f76cc1090f0 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc19f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50156/0 (socket says 172.21.15.46:50156) 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76cc10ac90 0x7f76cc1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 -- 172.21.15.46:0/3572657161 learned_addr learned my addr 172.21.15.46:0/3572657161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:05.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76caa00700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76cc10a2f0 0x7f76cc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:05.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 -- 172.21.15.46:0/3572657161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76cc10a2f0 msgr2=0x7f76cc19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:05.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76cc10a2f0 0x7f76cc19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 -- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76cc10ac90 msgr2=0x7f76cc1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76cc10ac90 0x7f76cc1a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.930+0000 7f76cb400700 1 -- 172.21.15.46:0/3572657161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc1a6fa0 con 0x7f76cc1090f0 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.931+0000 7f76cb400700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc19f340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00f690 tx=0x7f76bc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.931+0000 7f76c3e00700 1 -- 172.21.15.46:0/3572657161 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc026070 con 0x7f76cc1090f0 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.931+0000 7f76c3e00700 1 -- 172.21.15.46:0/3572657161 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76bc00fd60 con 0x7f76cc1090f0 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.931+0000 7f76c3e00700 1 -- 172.21.15.46:0/3572657161 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc017650 con 0x7f76cc1090f0 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.931+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76cc1a7230 con 0x7f76cc1090f0 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.931+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76cc1a7680 con 0x7f76cc1090f0 2024-04-17T07:58:05.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.932+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76cc069740 con 0x7f76cc1090f0 2024-04-17T07:58:05.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.934+0000 7f76c3e00700 1 -- 172.21.15.46:0/3572657161 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f76bc029070 con 0x7f76cc1090f0 2024-04-17T07:58:05.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.936+0000 7f76c3e00700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b00a3c40 0x7f76b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:05.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.936+0000 7f76c3e00700 1 -- 172.21.15.46:0/3572657161 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f76bc0d1a70 con 0x7f76cc1090f0 2024-04-17T07:58:05.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.936+0000 7f76caa00700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b00a3c40 0x7f76b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:05.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.937+0000 7f76caa00700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b00a3c40 0x7f76b00a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009c60 tx=0x7f76b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:05.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:05.938+0000 7f76c3e00700 1 -- 172.21.15.46:0/3572657161 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f76bc030080 con 0x7f76cc1090f0 2024-04-17T07:58:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.179+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f76cc062e90 con 0x7f76b00a3c40 2024-04-17T07:58:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.197+0000 7f76c3e00700 1 -- 172.21.15.46:0/3572657161 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f76cc062e90 con 0x7f76b00a3c40 2024-04-17T07:58:06.197 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-17T07:58:06.197 INFO:teuthology.orchestra.run.smithi046.stdout:mds.all 12/12 97s ago 3m smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi062=c;smithi062=f;smithi062=i;smithi062=l;count:12 2024-04-17T07:58:06.197 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 3/3 97s ago 9m smithi046=x;smithi053=y;smithi062=z;count:3 2024-04-17T07:58:06.197 INFO:teuthology.orchestra.run.smithi046.stdout:mon 3/3 97s ago 9m smithi046:172.21.15.46=a;smithi053:172.21.15.53=b;smithi062:172.21.15.62=c;count:3 2024-04-17T07:58:06.197 INFO:teuthology.orchestra.run.smithi046.stdout:osd 12 97s ago - 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b00a3c40 msgr2=0x7f76b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b00a3c40 0x7f76b00a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009c60 tx=0x7f76b8009260 comp rx=0 tx=0).stop 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 msgr2=0x7f76cc19f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc19f340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00f690 tx=0x7f76bc009580 comp rx=0 tx=0).stop 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 shutdown_connections 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b00a3c40 0x7f76b00a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76cc10ac90 0x7f76cc1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76cc1090f0 0x7f76cc19f340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 --2- 172.21.15.46:0/3572657161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76cc10a2f0 0x7f76cc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.200+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 >> 172.21.15.46:0/3572657161 conn(0x7f76cc0780c0 msgr2=0x7f76cc10d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.201+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 shutdown_connections 2024-04-17T07:58:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:06.201+0000 7f76d2000700 1 -- 172.21.15.46:0/3572657161 wait complete. 2024-04-17T07:58:07.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-17T07:58:07.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:07 smithi046 ceph-mon[32038]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:07.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:07 smithi046 ceph-mon[32038]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:07.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:07 smithi053 ceph-mon[36437]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:07.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:07 smithi053 ceph-mon[36437]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:07.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:07 smithi062 ceph-mon[36507]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:07.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:07 smithi062 ceph-mon[36507]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:07.680 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:58:09.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:09 smithi046 ceph-mon[32038]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:09.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:09 smithi053 ceph-mon[36437]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:09.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:09 smithi062 ceph-mon[36507]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 -- 172.21.15.46:0/1184755944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28340754b0 msgr2=0x7f2834075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 --2- 172.21.15.46:0/1184755944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28340754b0 0x7f2834075930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2824009990 tx=0x7f2824009ca0 comp rx=0 tx=0).stop 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 -- 172.21.15.46:0/1184755944 shutdown_connections 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 --2- 172.21.15.46:0/1184755944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f283410f090 0x7f2834111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 --2- 172.21.15.46:0/1184755944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2834076a80 0x7f2834074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 --2- 172.21.15.46:0/1184755944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28340754b0 0x7f2834075930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 -- 172.21.15.46:0/1184755944 >> 172.21.15.46:0/1184755944 conn(0x7f28340fe050 msgr2=0x7f2834100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.928+0000 7f283c800700 1 -- 172.21.15.46:0/1184755944 shutdown_connections 2024-04-17T07:58:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.929+0000 7f283c800700 1 -- 172.21.15.46:0/1184755944 wait complete. 2024-04-17T07:58:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.930+0000 7f283c800700 1 Processor -- start 2024-04-17T07:58:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.930+0000 7f283c800700 1 -- start start 2024-04-17T07:58:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f283c800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28340754b0 0x7f283419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f283c800700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834076a80 0x7f283419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f283c800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f283410f090 0x7f28341a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f283c800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2834114d10 con 0x7f28340754b0 2024-04-17T07:58:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f283c800700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2834114b90 con 0x7f2834076a80 2024-04-17T07:58:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f283c800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2834114e90 con 0x7f283410f090 2024-04-17T07:58:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f2839e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28340754b0 0x7f283419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f2839400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834076a80 0x7f283419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f2839400700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834076a80 0x7f283419f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40834/0 (socket says 172.21.15.46:40834) 2024-04-17T07:58:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f2839400700 1 -- 172.21.15.46:0/1024225582 learned_addr learned my addr 172.21.15.46:0/1024225582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.931+0000 7f283a800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f283410f090 0x7f28341a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f2839400700 1 -- 172.21.15.46:0/1024225582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f283410f090 msgr2=0x7f28341a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f2839400700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f283410f090 0x7f28341a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f2839400700 1 -- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28340754b0 msgr2=0x7f283419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f2839400700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28340754b0 0x7f283419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f2839400700 1 -- 172.21.15.46:0/1024225582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28341a7000 con 0x7f2834076a80 2024-04-17T07:58:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f283a800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f283410f090 0x7f28341a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:58:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f2839e00700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28340754b0 0x7f283419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f2839400700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834076a80 0x7f283419f8e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f282400b7b0 tx=0x7f282400bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f282ea00700 1 -- 172.21.15.46:0/1024225582 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824026070 con 0x7f2834076a80 2024-04-17T07:58:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28341a7290 con 0x7f2834076a80 2024-04-17T07:58:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f282ea00700 1 -- 172.21.15.46:0/1024225582 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f282400f460 con 0x7f2834076a80 2024-04-17T07:58:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f282ea00700 1 -- 172.21.15.46:0/1024225582 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824021650 con 0x7f2834076a80 2024-04-17T07:58:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.932+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28341a7740 con 0x7f2834076a80 2024-04-17T07:58:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.934+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2834069740 con 0x7f2834076a80 2024-04-17T07:58:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.936+0000 7f282ea00700 1 -- 172.21.15.46:0/1024225582 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2824029070 con 0x7f2834076a80 2024-04-17T07:58:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.937+0000 7f282ea00700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f281c0a3cc0 0x7f281c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.937+0000 7f282ea00700 1 -- 172.21.15.46:0/1024225582 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f28240d16e0 con 0x7f2834076a80 2024-04-17T07:58:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.938+0000 7f2839e00700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f281c0a3cc0 0x7f281c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.938+0000 7f2839e00700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f281c0a3cc0 0x7f281c0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2834076e60 tx=0x7f2828009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:09.940+0000 7f282ea00700 1 -- 172.21.15.46:0/1024225582 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2824013070 con 0x7f2834076a80 2024-04-17T07:58:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.178+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2834062f50 con 0x7f281c0a3cc0 2024-04-17T07:58:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.181+0000 7f282ea00700 1 -- 172.21.15.46:0/1024225582 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f2834062f50 con 0x7f281c0a3cc0 2024-04-17T07:58:10.180 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-04-17T07:58:10.180 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-04-17T07:58:10.180 INFO:teuthology.orchestra.run.smithi046.stdout:smithi053 172.21.15.53 2024-04-17T07:58:10.181 INFO:teuthology.orchestra.run.smithi046.stdout:smithi062 172.21.15.62 2024-04-17T07:58:10.181 INFO:teuthology.orchestra.run.smithi046.stdout:3 hosts in cluster 2024-04-17T07:58:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f281c0a3cc0 msgr2=0x7f281c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f281c0a3cc0 0x7f281c0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2834076e60 tx=0x7f2828009260 comp rx=0 tx=0).stop 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834076a80 msgr2=0x7f283419f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834076a80 0x7f283419f8e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f282400b7b0 tx=0x7f282400bbb0 comp rx=0 tx=0).stop 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 shutdown_connections 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f281c0a3cc0 0x7f281c0a6180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f283410f090 0x7f28341a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28340754b0 0x7f283419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 --2- 172.21.15.46:0/1024225582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834076a80 0x7f283419f8e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 >> 172.21.15.46:0/1024225582 conn(0x7f28340fe050 msgr2=0x7f2834102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 shutdown_connections 2024-04-17T07:58:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:10.184+0000 7f283c800700 1 -- 172.21.15.46:0/1024225582 wait complete. 2024-04-17T07:58:11.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-17T07:58:11.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:11 smithi046 ceph-mon[32038]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:11.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:11 smithi046 ceph-mon[32038]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:11 smithi053 ceph-mon[36437]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:11 smithi053 ceph-mon[36437]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:11.547 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:58:11.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:11 smithi062 ceph-mon[36507]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:11.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:11 smithi062 ceph-mon[36507]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:13.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:13 smithi053 ceph-mon[36437]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:13.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:13 smithi062 ceph-mon[36507]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:13.554 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:13 smithi046 ceph-mon[32038]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:13.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.919+0000 7fb196200700 1 -- 172.21.15.46:0/125375244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 msgr2=0x7fb19010ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:13.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.919+0000 7fb196200700 1 --2- 172.21.15.46:0/125375244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 0x7fb19010ae20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb18800b0a0 tx=0x7fb18800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:58:13.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.920+0000 7fb196200700 1 -- 172.21.15.46:0/125375244 shutdown_connections 2024-04-17T07:58:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.920+0000 7fb196200700 1 --2- 172.21.15.46:0/125375244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 0x7fb19010ae20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.920+0000 7fb196200700 1 --2- 172.21.15.46:0/125375244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1901029d0 0x7fb190102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.920+0000 7fb196200700 1 --2- 172.21.15.46:0/125375244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb190103bd0 0x7fb190104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.920+0000 7fb196200700 1 -- 172.21.15.46:0/125375244 >> 172.21.15.46:0/125375244 conn(0x7fb1900fdf90 msgr2=0x7fb1901003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.920+0000 7fb196200700 1 -- 172.21.15.46:0/125375244 shutdown_connections 2024-04-17T07:58:13.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.920+0000 7fb196200700 1 -- 172.21.15.46:0/125375244 wait complete. 2024-04-17T07:58:13.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 Processor -- start 2024-04-17T07:58:13.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 -- start start 2024-04-17T07:58:13.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1901029d0 0x7fb19019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:13.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb190103bd0 0x7fb19019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:13.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 0x7fb1901a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:13.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010e5f0 con 0x7fb190103bd0 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010e470 con 0x7fb1901029d0 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.921+0000 7fb196200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010e770 con 0x7fb190104570 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb18fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb190103bd0 0x7fb19019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb195800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 0x7fb1901a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb18fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb190103bd0 0x7fb19019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55944/0 (socket says 172.21.15.46:55944) 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb18fe00700 1 -- 172.21.15.46:0/223915928 learned_addr learned my addr 172.21.15.46:0/223915928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb194e00700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1901029d0 0x7fb19019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb195800700 1 -- 172.21.15.46:0/223915928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1901029d0 msgr2=0x7fb19019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb195800700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1901029d0 0x7fb19019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb195800700 1 -- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb190103bd0 msgr2=0x7fb19019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb195800700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb190103bd0 0x7fb19019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:13.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.922+0000 7fb195800700 1 -- 172.21.15.46:0/223915928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1901a2a60 con 0x7fb190104570 2024-04-17T07:58:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.923+0000 7fb195800700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 0x7fb1901a23c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb188007fd0 tx=0x7fb188004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.923+0000 7fb18d600700 1 -- 172.21.15.46:0/223915928 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188021070 con 0x7fb190104570 2024-04-17T07:58:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.923+0000 7fb18d600700 1 -- 172.21.15.46:0/223915928 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb188007b80 con 0x7fb190104570 2024-04-17T07:58:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.923+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1901a2cf0 con 0x7fb190104570 2024-04-17T07:58:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.923+0000 7fb18d600700 1 -- 172.21.15.46:0/223915928 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1880052a0 con 0x7fb190104570 2024-04-17T07:58:13.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.923+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a31a0 con 0x7fb190104570 2024-04-17T07:58:13.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.924+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb190069740 con 0x7fb190104570 2024-04-17T07:58:13.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.926+0000 7fb18d600700 1 -- 172.21.15.46:0/223915928 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb188024070 con 0x7fb190104570 2024-04-17T07:58:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.928+0000 7fb18d600700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1780a3cc0 0x7fb1780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.928+0000 7fb18d600700 1 -- 172.21.15.46:0/223915928 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb1880d1ab0 con 0x7fb190104570 2024-04-17T07:58:13.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.928+0000 7fb194e00700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1780a3cc0 0x7fb1780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:13.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.929+0000 7fb194e00700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1780a3cc0 0x7fb1780a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb190103a30 tx=0x7fb184009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:13.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:13.930+0000 7fb18d600700 1 -- 172.21.15.46:0/223915928 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb188007d30 con 0x7fb190104570 2024-04-17T07:58:14.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.171+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb1901a3430 con 0x7fb1780a3cc0 2024-04-17T07:58:14.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.175+0000 7fb18d600700 1 -- 172.21.15.46:0/223915928 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb1901a3430 con 0x7fb1780a3cc0 2024-04-17T07:58:14.175 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-17T07:58:14.175 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-04-17T07:58:14.175 INFO:teuthology.orchestra.run.smithi046.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-04-17T07:58:14.175 INFO:teuthology.orchestra.run.smithi046.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-17T07:58:14.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1780a3cc0 msgr2=0x7fb1780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1780a3cc0 0x7fb1780a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb190103a30 tx=0x7fb184009260 comp rx=0 tx=0).stop 2024-04-17T07:58:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 msgr2=0x7fb1901a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 0x7fb1901a23c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb188007fd0 tx=0x7fb188004f40 comp rx=0 tx=0).stop 2024-04-17T07:58:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 shutdown_connections 2024-04-17T07:58:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1780a3cc0 0x7fb1780a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb190104570 0x7fb1901a23c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1901029d0 0x7fb19019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 --2- 172.21.15.46:0/223915928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb190103bd0 0x7fb19019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.178+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 >> 172.21.15.46:0/223915928 conn(0x7fb1900fdf90 msgr2=0x7fb190074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.179+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 shutdown_connections 2024-04-17T07:58:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:14.179+0000 7fb196200700 1 -- 172.21.15.46:0/223915928 wait complete. 2024-04-17T07:58:15.039 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-17T07:58:15.048 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-04-17T07:58:15.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-17T07:58:15.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:15 smithi053 ceph-mon[36437]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:15.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:15 smithi053 ceph-mon[36437]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:58:15.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:15 smithi062 ceph-mon[36507]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:15.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:15 smithi062 ceph-mon[36507]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:15.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:15 smithi046 ceph-mon[32038]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:15.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:15 smithi046 ceph-mon[32038]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:58:16.215 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:16 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:58:16.215 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:16 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.215 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:16 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.215 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:16 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.215 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:16 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:16 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:58:16.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:16 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:16 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:16 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:16 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:16 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:58:16.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:16 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:16 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:16 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:16.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:16 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:17.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:17 smithi062 ceph-mon[36507]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:17.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:17.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:17 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:17.371 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:17 smithi053 ceph-mon[36437]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:17.371 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:17.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:17 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:17.583 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:17 smithi046 ceph-mon[32038]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:17.583 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:17.583 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:17 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1105416268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d01065a0 msgr2=0x7fa8d0106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1105416268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d01065a0 0x7fa8d0106a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800b0a0 tx=0x7fa8c800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:58:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1105416268 shutdown_connections 2024-04-17T07:58:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1105416268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d01065a0 0x7fa8d0106a20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1105416268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d0101260 0x7fa8d0103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1105416268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8d0103bd0 0x7fa8d0106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1105416268 >> 172.21.15.46:0/1105416268 conn(0x7fa8d00fce60 msgr2=0x7fa8d00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:17.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1105416268 shutdown_connections 2024-04-17T07:58:17.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.759+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1105416268 wait complete. 2024-04-17T07:58:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 Processor -- start 2024-04-17T07:58:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 -- start start 2024-04-17T07:58:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d0101260 0x7fa8d019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d0103bd0 0x7fa8d019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:17.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8d01065a0 0x7fa8d01a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0114b40 con 0x7fa8d0103bd0 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01149c0 con 0x7fa8d0101260 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.761+0000 7fa8cfe00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0114cc0 con 0x7fa8d01065a0 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d0101260 0x7fa8d019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d0101260 0x7fa8d019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:40860/0 (socket says 172.21.15.46:40860) 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 -- 172.21.15.46:0/1210632289 learned_addr learned my addr 172.21.15.46:0/1210632289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8ce000700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d0103bd0 0x7fa8d019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cf400700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8d01065a0 0x7fa8d01a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 -- 172.21.15.46:0/1210632289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8d01065a0 msgr2=0x7fa8d01a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8d01065a0 0x7fa8d01a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 -- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d0103bd0 msgr2=0x7fa8d019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d0103bd0 0x7fa8d019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.762+0000 7fa8cea00700 1 -- 172.21.15.46:0/1210632289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d01a6d50 con 0x7fa8d0101260 2024-04-17T07:58:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.763+0000 7fa8cea00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d0101260 0x7fa8d019f0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000e9a0 tx=0x7fa8c000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.763+0000 7fa8c7400700 1 -- 172.21.15.46:0/1210632289 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000cbd0 con 0x7fa8d0101260 2024-04-17T07:58:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.763+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d01a6fe0 con 0x7fa8d0101260 2024-04-17T07:58:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.763+0000 7fa8c7400700 1 -- 172.21.15.46:0/1210632289 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c000cd70 con 0x7fa8d0101260 2024-04-17T07:58:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.763+0000 7fa8c7400700 1 -- 172.21.15.46:0/1210632289 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00187a0 con 0x7fa8d0101260 2024-04-17T07:58:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.763+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d01a74e0 con 0x7fa8d0101260 2024-04-17T07:58:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.765+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d0069740 con 0x7fa8d0101260 2024-04-17T07:58:17.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.766+0000 7fa8c7400700 1 -- 172.21.15.46:0/1210632289 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa8c0010430 con 0x7fa8d0101260 2024-04-17T07:58:17.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.767+0000 7fa8c7400700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa8b40a3cc0 0x7fa8b40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.768+0000 7fa8c7400700 1 -- 172.21.15.46:0/1210632289 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa8c00d0970 con 0x7fa8d0101260 2024-04-17T07:58:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.768+0000 7fa8ce000700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa8b40a3cc0 0x7fa8b40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.769+0000 7fa8ce000700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa8b40a3cc0 0x7fa8b40a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009bf0 tx=0x7fa8bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:17.771+0000 7fa8c7400700 1 -- 172.21.15.46:0/1210632289 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa8c009cf60 con 0x7fa8d0101260 2024-04-17T07:58:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.060+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa8d0063660 con 0x7fa8d0101260 2024-04-17T07:58:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.061+0000 7fa8c7400700 1 -- 172.21.15.46:0/1210632289 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3051 (secure 0 0 0) 0x7fa8c009c860 con 0x7fa8d0101260 2024-04-17T07:58:18.062 INFO:teuthology.orchestra.run.smithi046.stdout:e23 2024-04-17T07:58:18.062 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T07:58:18.062 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 23 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-04-17T07:55:26.016968+0000 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-04-17T07:55:54.519826+0000 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 300 2024-04-17T07:58:18.063 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:snap_rstat 1 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 3 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1,2 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=24596,1=24556,2=24542} 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-04-17T07:58:18.064 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3,4] 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 24596 members: 24596,24556,24542 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b{0:24596} state up:active seq 3 addr [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.i{0:24538} state up:standby-replay seq 1 addr [v2:172.21.15.62:6838/665634381,v1:172.21.15.62:6839/665634381] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.g{1:24556} state up:active seq 12 addr [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c{1:24517} state up:standby-replay seq 1 addr [v2:172.21.15.62:6836/956269488,v1:172.21.15.62:6837/956269488] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.h{2:24542} state up:active seq 14 addr [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f{2:24505} state up:standby-replay seq 1 addr [v2:172.21.15.62:6834/2191814520,v1:172.21.15.62:6835/2191814520] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.065 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.j{-1:14379} state up:standby seq 1 addr [v2:172.21.15.46:6834/1217684239,v1:172.21.15.46:6835/1217684239] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d{-1:14391} state up:standby seq 1 addr [v2:172.21.15.46:6836/1258153884,v1:172.21.15.46:6837/1258153884] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a{-1:14460} state up:standby seq 1 addr [v2:172.21.15.46:6840/1599229052,v1:172.21.15.46:6841/1599229052] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.62:6832/1314622841,v1:172.21.15.62:6833/1314622841] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.k{-1:24461} state up:standby seq 1 addr [v2:172.21.15.53:6832/2557309272,v1:172.21.15.53:6833/2557309272] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.066 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e{-1:24503} state up:standby seq 1 addr [v2:172.21.15.53:6834/3940284604,v1:172.21.15.53:6835/3940284604] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:58:18.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa8b40a3cc0 msgr2=0x7fa8b40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:18.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa8b40a3cc0 0x7fa8b40a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009bf0 tx=0x7fa8bc009260 comp rx=0 tx=0).stop 2024-04-17T07:58:18.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d0101260 msgr2=0x7fa8d019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:18.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d0101260 0x7fa8d019f0f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000e9a0 tx=0x7fa8c000ecb0 comp rx=0 tx=0).stop 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 shutdown_connections 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa8b40a3cc0 0x7fa8b40a6180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8d01065a0 0x7fa8d01a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8d0101260 0x7fa8d019f0f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 --2- 172.21.15.46:0/1210632289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d0103bd0 0x7fa8d019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 >> 172.21.15.46:0/1210632289 conn(0x7fa8d00fce60 msgr2=0x7fa8d0101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 shutdown_connections 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:18.064+0000 7fa8cfe00700 1 -- 172.21.15.46:0/1210632289 wait complete. 2024-04-17T07:58:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-04-17T07:58:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:18 smithi053 ceph-mon[36437]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:18 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:18 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:18 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:18 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:18 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1210632289' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T07:58:19.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:18 smithi062 ceph-mon[36507]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:19.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:18 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:18 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:18 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:18 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:18 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1210632289' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T07:58:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:18 smithi046 ceph-mon[32038]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:18 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:18 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:18 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:18 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:19.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:18 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1210632289' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T07:58:19.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-17T07:58:20.011 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:58:21.143 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:20 smithi046 ceph-mon[32038]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:21.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:20 smithi053 ceph-mon[36437]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:21.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:20 smithi062 ceph-mon[36507]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 -- 172.21.15.46:0/2214100899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc102ae0 msgr2=0x7f2bdc102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 --2- 172.21.15.46:0/2214100899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc102f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00099c0 tx=0x7f2bd0009cd0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 -- 172.21.15.46:0/2214100899 shutdown_connections 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 --2- 172.21.15.46:0/2214100899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc104680 0x7f2bdc10af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 --2- 172.21.15.46:0/2214100899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc102f00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 --2- 172.21.15.46:0/2214100899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bdc103ce0 0x7f2bdc104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 -- 172.21.15.46:0/2214100899 >> 172.21.15.46:0/2214100899 conn(0x7f2bdc0fe0a0 msgr2=0x7f2bdc1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 -- 172.21.15.46:0/2214100899 shutdown_connections 2024-04-17T07:58:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.443+0000 7f2be3a00700 1 -- 172.21.15.46:0/2214100899 wait complete. 2024-04-17T07:58:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.444+0000 7f2be3a00700 1 Processor -- start 2024-04-17T07:58:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.444+0000 7f2be3a00700 1 -- start start 2024-04-17T07:58:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be3a00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be3a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc103ce0 0x7f2bdc19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be3a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bdc104680 0x7f2bdc1a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be3a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc10e6a0 con 0x7f2bdc103ce0 2024-04-17T07:58:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be3a00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc10e520 con 0x7f2bdc102ae0 2024-04-17T07:58:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be3a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc10e820 con 0x7f2bdc104680 2024-04-17T07:58:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2bdbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc103ce0 0x7f2bdc19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be1000700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2bdbe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc103ce0 0x7f2bdc19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57282/0 (socket says 172.21.15.46:57282) 2024-04-17T07:58:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2bdbe00700 1 -- 172.21.15.46:0/1824392969 learned_addr learned my addr 172.21.15.46:0/1824392969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.445+0000 7f2be1a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bdc104680 0x7f2bdc1a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2be1000700 1 -- 172.21.15.46:0/1824392969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bdc104680 msgr2=0x7f2bdc1a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2be1000700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bdc104680 0x7f2bdc1a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2be1000700 1 -- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc103ce0 msgr2=0x7f2bdc19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2be1000700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc103ce0 0x7f2bdc19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2be1000700 1 -- 172.21.15.46:0/1824392969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc1a2b20 con 0x7f2bdc102ae0 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2be1a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bdc104680 0x7f2bdc1a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2bdbe00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc103ce0 0x7f2bdc19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.446+0000 7f2be1000700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc19aec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00098e0 tx=0x7f2bd000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.447+0000 7f2bd9600700 1 -- 172.21.15.46:0/1824392969 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0009e60 con 0x7f2bdc102ae0 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.447+0000 7f2bd9600700 1 -- 172.21.15.46:0/1824392969 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0003e30 con 0x7f2bdc102ae0 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.447+0000 7f2bd9600700 1 -- 172.21.15.46:0/1824392969 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000fd90 con 0x7f2bdc102ae0 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.447+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc1a2db0 con 0x7f2bdc102ae0 2024-04-17T07:58:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.447+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a31f0 con 0x7f2bdc102ae0 2024-04-17T07:58:22.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.448+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc1a3420 con 0x7f2bdc102ae0 2024-04-17T07:58:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.450+0000 7f2bd9600700 1 -- 172.21.15.46:0/1824392969 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2bd000f4f0 con 0x7f2bdc102ae0 2024-04-17T07:58:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.451+0000 7f2bd9600700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc40a3cc0 0x7f2bc40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.451+0000 7f2bd9600700 1 -- 172.21.15.46:0/1824392969 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f2bd00d1410 con 0x7f2bdc102ae0 2024-04-17T07:58:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.452+0000 7f2bdbe00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc40a3cc0 0x7f2bc40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.452+0000 7f2bdbe00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc40a3cc0 0x7f2bc40a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8011550 tx=0x7f2bc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:22.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.454+0000 7f2bd9600700 1 -- 172.21.15.46:0/1824392969 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2bd0013070 con 0x7f2bdc102ae0 2024-04-17T07:58:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.687+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2bdc1a3760 con 0x7f2bdc102ae0 2024-04-17T07:58:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.688+0000 7f2bd9600700 1 -- 172.21.15.46:0/1824392969 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5602 (secure 0 0 0) 0x7f2bd009da90 con 0x7f2bdc102ae0 2024-04-17T07:58:22.688 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 93 2024-04-17T07:58:22.688 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T07:58:22.688 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-04-17T07:46:11.976719+0000 2024-04-17T07:58:22.688 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-04-17T07:57:22.972449+0000 2024-04-17T07:58:22.688 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-17T07:58:22.688 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 27 2024-04-17T07:58:22.688 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client luminous 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release squid 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 12 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6802/3982963402,v1:172.21.15.46:6803/3982963402] [v2:172.21.15.46:6804/3982963402,v1:172.21.15.46:6805/3982963402] exists,up 437464e9-4b20-4de4-9782-89a0daec7f70 2024-04-17T07:58:22.689 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/445046635,v1:172.21.15.53:6801/445046635] [v2:172.21.15.53:6802/445046635,v1:172.21.15.53:6803/445046635] exists,up d0511b8c-f523-4678-9683-ceaa6a1e65d1 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6800/3705420480,v1:172.21.15.62:6801/3705420480] [v2:172.21.15.62:6802/3705420480,v1:172.21.15.62:6803/3705420480] exists,up d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/24338866,v1:172.21.15.46:6811/24338866] [v2:172.21.15.46:6812/24338866,v1:172.21.15.46:6813/24338866] exists,up 398d84f2-695b-4446-872d-55ba26ab8c94 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 33 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/3909086430,v1:172.21.15.53:6809/3909086430] [v2:172.21.15.53:6810/3909086430,v1:172.21.15.53:6811/3909086430] exists,up 5fb57543-bfba-422e-9306-4b1b50a98c0b 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6808/2113355560,v1:172.21.15.62:6809/2113355560] [v2:172.21.15.62:6810/2113355560,v1:172.21.15.62:6811/2113355560] exists,up 26424544-5d91-45e4-8e62-00945c375bb5 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6818/1575836567,v1:172.21.15.46:6819/1575836567] [v2:172.21.15.46:6820/1575836567,v1:172.21.15.46:6821/1575836567] exists,up 720d8bdb-9a6a-4fcc-bda7-b575a895ea17 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/3581494235,v1:172.21.15.53:6817/3581494235] [v2:172.21.15.53:6818/3581494235,v1:172.21.15.53:6819/3581494235] exists,up 7b7d7a00-52ea-47ab-b844-d5a192861516 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6816/520671268,v1:172.21.15.62:6817/520671268] [v2:172.21.15.62:6818/520671268,v1:172.21.15.62:6819/520671268] exists,up 0f936f20-269a-496b-af26-a962f09c26a3 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6826/1655693057,v1:172.21.15.46:6827/1655693057] [v2:172.21.15.46:6828/1655693057,v1:172.21.15.46:6829/1655693057] exists,up 31f44d01-ea3b-4b7a-a763-8d39e757395b 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/795360717,v1:172.21.15.53:6825/795360717] [v2:172.21.15.53:6826/795360717,v1:172.21.15.53:6827/795360717] exists,up 93b1b906-94d1-4af6-8ab1-b69157e8185c 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6824/359287884,v1:172.21.15.62:6825/359287884] [v2:172.21.15.62:6826/359287884,v1:172.21.15.62:6827/359287884] exists,up aaf98df8-8674-48c4-87d5-531ef5a44098 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.0 [8,4] 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.b [2,4] 2024-04-17T07:58:22.690 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.e [2,1] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.11 [7,11] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.1d [8,11] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.3 [4,9] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.13 [4,9] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1e [3,9] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.2 [9,1] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.5 [5,8] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.6 [9,11] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 4.16 [9,8] 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/461113542 expires 2024-04-18T07:47:21.351556+0000 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/461113542 expires 2024-04-18T07:47:21.351556+0000 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/663006341 expires 2024-04-18T07:47:21.351556+0000 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/63594551 expires 2024-04-18T07:46:45.851840+0000 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/61074341 expires 2024-04-18T07:47:21.351556+0000 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1435664148 expires 2024-04-18T07:46:45.851840+0000 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1911895676 expires 2024-04-18T07:47:21.351556+0000 2024-04-17T07:58:22.691 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/4094803685 expires 2024-04-18T07:46:45.851840+0000 2024-04-17T07:58:22.692 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2289010020 expires 2024-04-18T07:46:45.851840+0000 2024-04-17T07:58:22.692 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/4094803685 expires 2024-04-18T07:46:45.851840+0000 2024-04-17T07:58:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc40a3cc0 msgr2=0x7f2bc40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc40a3cc0 0x7f2bc40a6180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8011550 tx=0x7f2bc8009260 comp rx=0 tx=0).stop 2024-04-17T07:58:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc102ae0 msgr2=0x7f2bdc19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc19aec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00098e0 tx=0x7f2bd000bf60 comp rx=0 tx=0).stop 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 shutdown_connections 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc40a3cc0 0x7f2bc40a6180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bdc104680 0x7f2bdc1a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc102ae0 0x7f2bdc19aec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 --2- 172.21.15.46:0/1824392969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bdc103ce0 0x7f2bdc19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.691+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 >> 172.21.15.46:0/1824392969 conn(0x7f2bdc0fe0a0 msgr2=0x7f2bdc074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.692+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 shutdown_connections 2024-04-17T07:58:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:22.692+0000 7f2be3a00700 1 -- 172.21.15.46:0/1824392969 wait complete. 2024-04-17T07:58:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:23 smithi053 ceph-mon[36437]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:23 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:23 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:23.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:23 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1824392969' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T07:58:23.484 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:23 smithi046 ceph-mon[32038]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:23.484 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:23 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:23.484 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:23 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:23.484 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:23 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1824392969' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T07:58:23.541 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-17T07:58:23.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:58:23.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:23 smithi062 ceph-mon[36507]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:23.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:23 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:23.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:23 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:23.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:23 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1824392969' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T07:58:23.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 -- 172.21.15.46:0/443964666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c410a060 msgr2=0x7fc8c41108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/443964666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c410a060 0x7fc8c41108f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009ca0 tx=0x7fc8bc02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 -- 172.21.15.46:0/443964666 shutdown_connections 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/443964666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c410a060 0x7fc8c41108f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/443964666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8c41096a0 0x7fc8c4109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/443964666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4108500 0x7fc8c4108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 -- 172.21.15.46:0/443964666 >> 172.21.15.46:0/443964666 conn(0x7fc8c4077450 msgr2=0x7fc8c4079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 -- 172.21.15.46:0/443964666 shutdown_connections 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.765+0000 7fc8c8c00640 1 -- 172.21.15.46:0/443964666 wait complete. 2024-04-17T07:58:23.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 Processor -- start 2024-04-17T07:58:23.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 -- start start 2024-04-17T07:58:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8c4108500 0x7fc8c419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c41096a0 0x7fc8c419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c410a060 0x7fc8c41a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c4114110 con 0x7fc8c410a060 2024-04-17T07:58:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c4113f90 con 0x7fc8c4108500 2024-04-17T07:58:23.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.767+0000 7fc8c8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c4114290 con 0x7fc8c41096a0 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c41096a0 0x7fc8c419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c41096a0 0x7fc8c419ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50106/0 (socket says 172.21.15.46:50106) 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 -- 172.21.15.46:0/3641471737 learned_addr learned my addr 172.21.15.46:0/3641471737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8c3400640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8c4108500 0x7fc8c419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8c3e00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c410a060 0x7fc8c41a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 -- 172.21.15.46:0/3641471737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8c4108500 msgr2=0x7fc8c419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8c4108500 0x7fc8c419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 -- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c410a060 msgr2=0x7fc8c41a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c410a060 0x7fc8c41a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8bbe00640 1 -- 172.21.15.46:0/3641471737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c41a62f0 con 0x7fc8c41096a0 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.768+0000 7fc8c3e00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c410a060 0x7fc8c41a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:58:23.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.769+0000 7fc8bbe00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c41096a0 0x7fc8c419ec10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00e9f0 tx=0x7fc8ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.769+0000 7fc8c0c00640 1 -- 172.21.15.46:0/3641471737 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac00cd80 con 0x7fc8c41096a0 2024-04-17T07:58:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.769+0000 7fc8c0c00640 1 -- 172.21.15.46:0/3641471737 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8ac010910 con 0x7fc8c41096a0 2024-04-17T07:58:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.769+0000 7fc8c0c00640 1 -- 172.21.15.46:0/3641471737 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac018990 con 0x7fc8c41096a0 2024-04-17T07:58:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.769+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c41a65e0 con 0x7fc8c41096a0 2024-04-17T07:58:23.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.769+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c41a6b80 con 0x7fc8c41096a0 2024-04-17T07:58:23.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.770+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c410f770 con 0x7fc8c41096a0 2024-04-17T07:58:23.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.772+0000 7fc8c0c00640 1 -- 172.21.15.46:0/3641471737 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc8ac010ac0 con 0x7fc8c41096a0 2024-04-17T07:58:23.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.774+0000 7fc8c0c00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8a80a3c20 0x7fc8a80a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:23.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.774+0000 7fc8c0c00640 1 -- 172.21.15.46:0/3641471737 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc8ac0d0b10 con 0x7fc8c41096a0 2024-04-17T07:58:23.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.774+0000 7fc8c3400640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8a80a3c20 0x7fc8a80a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:23.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.775+0000 7fc8c3400640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8a80a3c20 0x7fc8a80a60e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40095f0 tx=0x7fc8b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:23.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.778+0000 7fc8c0c00640 1 -- 172.21.15.46:0/3641471737 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc8ac09d100 con 0x7fc8c41096a0 2024-04-17T07:58:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.991+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8c4062b60 con 0x7fc8c41096a0 2024-04-17T07:58:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.994+0000 7fc8c0c00640 1 -- 172.21.15.46:0/3641471737 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7fc8ac09ca00 con 0x7fc8c41096a0 2024-04-17T07:58:23.994 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:23.994 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8a80a3c20 msgr2=0x7fc8a80a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8a80a3c20 0x7fc8a80a60e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40095f0 tx=0x7fc8b4005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c41096a0 msgr2=0x7fc8c419ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c41096a0 0x7fc8c419ec10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00e9f0 tx=0x7fc8ac00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 shutdown_connections 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8a80a3c20 0x7fc8a80a60e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c410a060 0x7fc8c41a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8c41096a0 0x7fc8c419ec10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 --2- 172.21.15.46:0/3641471737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8c4108500 0x7fc8c419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 >> 172.21.15.46:0/3641471737 conn(0x7fc8c4077450 msgr2=0x7fc8c410c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 shutdown_connections 2024-04-17T07:58:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:23.997+0000 7fc8c8c00640 1 -- 172.21.15.46:0/3641471737 wait complete. 2024-04-17T07:58:24.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:58:24.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:58:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 -- 172.21.15.46:0/2499850138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581095b0 msgr2=0x7fb458109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 --2- 172.21.15.46:0/2499850138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581095b0 0x7fb458109a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb444009900 tx=0x7fb44402bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 -- 172.21.15.46:0/2499850138 shutdown_connections 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 --2- 172.21.15.46:0/2499850138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb458109f70 0x7fb458110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 --2- 172.21.15.46:0/2499850138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581095b0 0x7fb458109a30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 --2- 172.21.15.46:0/2499850138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4581083c0 0x7fb4581087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 -- 172.21.15.46:0/2499850138 >> 172.21.15.46:0/2499850138 conn(0x7fb4580773a0 msgr2=0x7fb4580797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 -- 172.21.15.46:0/2499850138 shutdown_connections 2024-04-17T07:58:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.253+0000 7fb457e00640 1 -- 172.21.15.46:0/2499850138 wait complete. 2024-04-17T07:58:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.254+0000 7fb457e00640 1 Processor -- start 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.254+0000 7fb457e00640 1 -- start start 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.254+0000 7fb457e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581083c0 0x7fb45819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.254+0000 7fb457e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4581095b0 0x7fb45819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb457e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb458109f70 0x7fb4581a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb457e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4581140a0 con 0x7fb4581095b0 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb457e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb458113f20 con 0x7fb458109f70 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb457e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb458114220 con 0x7fb4581083c0 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4581095b0 0x7fb45819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4581095b0 0x7fb45819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57316/0 (socket says 172.21.15.46:57316) 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 -- 172.21.15.46:0/3545774114 learned_addr learned my addr 172.21.15.46:0/3545774114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb457400640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb458109f70 0x7fb4581a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456a00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581083c0 0x7fb45819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 -- 172.21.15.46:0/3545774114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581083c0 msgr2=0x7fb45819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581083c0 0x7fb45819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 -- 172.21.15.46:0/3545774114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb458109f70 msgr2=0x7fb4581a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb458109f70 0x7fb4581a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb456000640 1 -- 172.21.15.46:0/3545774114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4581a62e0 con 0x7fb4581095b0 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.255+0000 7fb457400640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb458109f70 0x7fb4581a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.256+0000 7fb456000640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4581095b0 0x7fb45819ec00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb44402c8f0 tx=0x7fb444005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.256+0000 7fb44f400640 1 -- 172.21.15.46:0/3545774114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444041070 con 0x7fb4581095b0 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.256+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4581a6570 con 0x7fb4581095b0 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.256+0000 7fb44f400640 1 -- 172.21.15.46:0/3545774114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4440044d0 con 0x7fb4581095b0 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.256+0000 7fb44f400640 1 -- 172.21.15.46:0/3545774114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444034440 con 0x7fb4581095b0 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.256+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4581a6ab0 con 0x7fb4581095b0 2024-04-17T07:58:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.258+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45810f680 con 0x7fb4581095b0 2024-04-17T07:58:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.260+0000 7fb44f400640 1 -- 172.21.15.46:0/3545774114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb444044070 con 0x7fb4581095b0 2024-04-17T07:58:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.261+0000 7fb44f400640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb43c0a3ba0 0x7fb43c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.261+0000 7fb44f400640 1 -- 172.21.15.46:0/3545774114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb4440f2310 con 0x7fb4581095b0 2024-04-17T07:58:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.261+0000 7fb456a00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb43c0a3ba0 0x7fb43c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.262+0000 7fb456a00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb43c0a3ba0 0x7fb43c0a6060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb458109410 tx=0x7fb448005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.266+0000 7fb44f400640 1 -- 172.21.15.46:0/3545774114 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb4440be2b0 con 0x7fb4581095b0 2024-04-17T07:58:24.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:58:24.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:58:24.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:24.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:24 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3641471737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:24.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:58:24.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:58:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:24.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:24 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3641471737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:24.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.477+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb458062b60 con 0x7fb4581095b0 2024-04-17T07:58:24.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.478+0000 7fb44f400640 1 -- 172.21.15.46:0/3545774114 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb4440b8e20 con 0x7fb4581095b0 2024-04-17T07:58:24.477 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:58:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb43c0a3ba0 msgr2=0x7fb43c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb43c0a3ba0 0x7fb43c0a6060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb458109410 tx=0x7fb448005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4581095b0 msgr2=0x7fb45819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4581095b0 0x7fb45819ec00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb44402c8f0 tx=0x7fb444005d50 comp rx=0 tx=0).stop 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 shutdown_connections 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb43c0a3ba0 0x7fb43c0a6060 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb458109f70 0x7fb4581a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4581095b0 0x7fb45819ec00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 --2- 172.21.15.46:0/3545774114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4581083c0 0x7fb45819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 >> 172.21.15.46:0/3545774114 conn(0x7fb4580773a0 msgr2=0x7fb45810c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 shutdown_connections 2024-04-17T07:58:24.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.481+0000 7fb457e00640 1 -- 172.21.15.46:0/3545774114 wait complete. 2024-04-17T07:58:24.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:58:24.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:58:24.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:58:24.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:58:24.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:24 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3641471737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 -- 172.21.15.46:0/3631987122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 msgr2=0x7f738c10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 --2- 172.21.15.46:0/3631987122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 0x7f738c10c600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7384009ca0 tx=0x7f738402c100 comp rx=0 tx=0).stop 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 -- 172.21.15.46:0/3631987122 shutdown_connections 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 --2- 172.21.15.46:0/3631987122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 0x7f738c10c600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 --2- 172.21.15.46:0/3631987122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1053b0 0x7f738c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 --2- 172.21.15.46:0/3631987122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f738c1041c0 0x7f738c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 -- 172.21.15.46:0/3631987122 >> 172.21.15.46:0/3631987122 conn(0x7f738c0ff750 msgr2=0x7f738c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 -- 172.21.15.46:0/3631987122 shutdown_connections 2024-04-17T07:58:24.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.727+0000 7f7392e00640 1 -- 172.21.15.46:0/3631987122 wait complete. 2024-04-17T07:58:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.729+0000 7f7392e00640 1 Processor -- start 2024-04-17T07:58:24.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.729+0000 7f7392e00640 1 -- start start 2024-04-17T07:58:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.729+0000 7f7392e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1041c0 0x7f738c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.729+0000 7f7392e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f738c1053b0 0x7f738c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.729+0000 7f7392e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 0x7f738c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f7392e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f738c10fe50 con 0x7f738c1041c0 2024-04-17T07:58:24.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f7392e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f738c10fcd0 con 0x7f738c1053b0 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f7392e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f738c10ffd0 con 0x7f738c105d70 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1041c0 0x7f738c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1041c0 0x7f738c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57340/0 (socket says 172.21.15.46:57340) 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 -- 172.21.15.46:0/987321901 learned_addr learned my addr 172.21.15.46:0/987321901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f7390e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 0x7f738c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f7383400640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f738c1053b0 0x7f738c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 -- 172.21.15.46:0/987321901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 msgr2=0x7f738c1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 0x7f738c1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 -- 172.21.15.46:0/987321901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f738c1053b0 msgr2=0x7f738c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f738c1053b0 0x7f738c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f738be00640 1 -- 172.21.15.46:0/987321901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c1a20b0 con 0x7f738c1041c0 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f7383400640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f738c1053b0 0x7f738c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.730+0000 7f7390e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 0x7f738c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.731+0000 7f738be00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1041c0 0x7f738c19a490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e9c0 tx=0x7f737c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.731+0000 7f7389600640 1 -- 172.21.15.46:0/987321901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c00cd80 con 0x7f738c1041c0 2024-04-17T07:58:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.731+0000 7f7389600640 1 -- 172.21.15.46:0/987321901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f737c0035f0 con 0x7f738c1041c0 2024-04-17T07:58:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.731+0000 7f7389600640 1 -- 172.21.15.46:0/987321901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c010690 con 0x7f738c1041c0 2024-04-17T07:58:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.731+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c1a23a0 con 0x7f738c1041c0 2024-04-17T07:58:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.731+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c1a28f0 con 0x7f738c1041c0 2024-04-17T07:58:24.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.732+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738c10b480 con 0x7f738c1041c0 2024-04-17T07:58:24.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.735+0000 7f7389600640 1 -- 172.21.15.46:0/987321901 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f737c0037a0 con 0x7f738c1041c0 2024-04-17T07:58:24.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.736+0000 7f7389600640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f73700a3c30 0x7f73700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:24.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.736+0000 7f7389600640 1 -- 172.21.15.46:0/987321901 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f737c0d0ad0 con 0x7f738c1041c0 2024-04-17T07:58:24.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.736+0000 7f7383400640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f73700a3c30 0x7f73700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:24.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.737+0000 7f7383400640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f73700a3c30 0x7f73700a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f738c19b9a0 tx=0x7f7374005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:24.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.741+0000 7f7389600640 1 -- 172.21.15.46:0/987321901 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f737c09d0c0 con 0x7f738c1041c0 2024-04-17T07:58:24.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.948+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f738c062b60 con 0x7f738c1041c0 2024-04-17T07:58:24.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.949+0000 7f7389600640 1 -- 172.21.15.46:0/987321901 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f737c09c9c0 con 0x7f738c1041c0 2024-04-17T07:58:24.948 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:24.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f73700a3c30 msgr2=0x7f73700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f73700a3c30 0x7f73700a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f738c19b9a0 tx=0x7f7374005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1041c0 msgr2=0x7f738c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1041c0 0x7f738c19a490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f737c00e9c0 tx=0x7f737c00ee80 comp rx=0 tx=0).stop 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 shutdown_connections 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f73700a3c30 0x7f73700a60f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f738c105d70 0x7f738c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f738c1053b0 0x7f738c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 --2- 172.21.15.46:0/987321901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f738c1041c0 0x7f738c19a490 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 >> 172.21.15.46:0/987321901 conn(0x7f738c0ff750 msgr2=0x7f738c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 shutdown_connections 2024-04-17T07:58:24.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:24.953+0000 7f7392e00640 1 -- 172.21.15.46:0/987321901 wait complete. 2024-04-17T07:58:24.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:58:25.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 -- 172.21.15.46:0/876875698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a540749c0 msgr2=0x7f7a54074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 --2- 172.21.15.46:0/876875698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a540749c0 0x7f7a54074e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009900 tx=0x7f7a3c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 -- 172.21.15.46:0/876875698 shutdown_connections 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 --2- 172.21.15.46:0/876875698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a54109c70 0x7f7a5410c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 --2- 172.21.15.46:0/876875698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a540749c0 0x7f7a54074e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 --2- 172.21.15.46:0/876875698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a54075fa0 0x7f7a540743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 -- 172.21.15.46:0/876875698 >> 172.21.15.46:0/876875698 conn(0x7f7a540fd320 msgr2=0x7f7a540ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 -- 172.21.15.46:0/876875698 shutdown_connections 2024-04-17T07:58:25.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.201+0000 7f7a59c00640 1 -- 172.21.15.46:0/876875698 wait complete. 2024-04-17T07:58:25.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.202+0000 7f7a59c00640 1 Processor -- start 2024-04-17T07:58:25.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.202+0000 7f7a59c00640 1 -- start start 2024-04-17T07:58:25.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a59c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a540749c0 0x7f7a5419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a59c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a54075fa0 0x7f7a5419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a59c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a54109c70 0x7f7a541a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a59c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410fc30 con 0x7f7a54109c70 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a59c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410fab0 con 0x7f7a540749c0 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a59c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410fdb0 con 0x7f7a54075fa0 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a540749c0 0x7f7a5419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a53e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a540749c0 0x7f7a5419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46736/0 (socket says 172.21.15.46:46736) 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a53e00640 1 -- 172.21.15.46:0/3771478283 learned_addr learned my addr 172.21.15.46:0/3771478283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a59200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a54109c70 0x7f7a541a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.203+0000 7f7a4b400640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a54075fa0 0x7f7a5419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a53e00640 1 -- 172.21.15.46:0/3771478283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a54075fa0 msgr2=0x7f7a5419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a53e00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a54075fa0 0x7f7a5419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a53e00640 1 -- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a54109c70 msgr2=0x7f7a541a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a53e00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a54109c70 0x7f7a541a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a53e00640 1 -- 172.21.15.46:0/3771478283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a541a1e50 con 0x7f7a540749c0 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a53e00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a540749c0 0x7f7a5419a230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400ea60 tx=0x7f7a4400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a51600640 1 -- 172.21.15.46:0/3771478283 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400cde0 con 0x7f7a540749c0 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a541a2140 con 0x7f7a540749c0 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a51600640 1 -- 172.21.15.46:0/3771478283 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a440035f0 con 0x7f7a540749c0 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a51600640 1 -- 172.21.15.46:0/3771478283 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44010670 con 0x7f7a540749c0 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.204+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a541a26b0 con 0x7f7a540749c0 2024-04-17T07:58:25.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.206+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a541089a0 con 0x7f7a540749c0 2024-04-17T07:58:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.208+0000 7f7a51600640 1 -- 172.21.15.46:0/3771478283 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7a440037a0 con 0x7f7a540749c0 2024-04-17T07:58:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.209+0000 7f7a51600640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7a380a3ba0 0x7f7a380a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.209+0000 7f7a51600640 1 -- 172.21.15.46:0/3771478283 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7a440d0d40 con 0x7f7a540749c0 2024-04-17T07:58:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.210+0000 7f7a4b400640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7a380a3ba0 0x7f7a380a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.210+0000 7f7a4b400640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7a380a3ba0 0x7f7a380a6060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c006010 tx=0x7f7a3c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:25.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.212+0000 7f7a51600640 1 -- 172.21.15.46:0/3771478283 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7a4409d330 con 0x7f7a540749c0 2024-04-17T07:58:25.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:25 smithi046 ceph-mon[32038]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:25.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:25 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3545774114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:25.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:25 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/987321901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:25.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.421+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a5402d5d0 con 0x7f7a540749c0 2024-04-17T07:58:25.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.422+0000 7f7a51600640 1 -- 172.21.15.46:0/3771478283 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f7a4409cc30 con 0x7f7a540749c0 2024-04-17T07:58:25.421 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.424+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7a380a3ba0 msgr2=0x7f7a380a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.424+0000 7f7a59c00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7a380a3ba0 0x7f7a380a6060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c006010 tx=0x7f7a3c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.424+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a540749c0 msgr2=0x7f7a5419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.424+0000 7f7a59c00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a540749c0 0x7f7a5419a230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400ea60 tx=0x7f7a4400ef20 comp rx=0 tx=0).stop 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 shutdown_connections 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7a380a3ba0 0x7f7a380a6060 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a54109c70 0x7f7a541a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a54075fa0 0x7f7a5419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 --2- 172.21.15.46:0/3771478283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a540749c0 0x7f7a5419a230 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 >> 172.21.15.46:0/3771478283 conn(0x7f7a540fd320 msgr2=0x7f7a54105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 shutdown_connections 2024-04-17T07:58:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.425+0000 7f7a59c00640 1 -- 172.21.15.46:0/3771478283 wait complete. 2024-04-17T07:58:25.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:58:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:25 smithi053 ceph-mon[36437]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:25 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3545774114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:25.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:25 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/987321901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:25.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:25 smithi062 ceph-mon[36507]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:25.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:25 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3545774114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:25.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:25 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/987321901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 -- 172.21.15.46:0/2827839254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4107ab0 msgr2=0x7f34a4109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 --2- 172.21.15.46:0/2827839254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4107ab0 0x7f34a4109f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f349c009ca0 tx=0x7f349c02c100 comp rx=0 tx=0).stop 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 -- 172.21.15.46:0/2827839254 shutdown_connections 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 --2- 172.21.15.46:0/2827839254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4107ab0 0x7f34a4109f00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 --2- 172.21.15.46:0/2827839254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4105180 0x7f34a4107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 --2- 172.21.15.46:0/2827839254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a4102850 0x7f34a4104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 -- 172.21.15.46:0/2827839254 >> 172.21.15.46:0/2827839254 conn(0x7f34a40fc240 msgr2=0x7f34a40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 -- 172.21.15.46:0/2827839254 shutdown_connections 2024-04-17T07:58:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.667+0000 7f34aa800640 1 -- 172.21.15.46:0/2827839254 wait complete. 2024-04-17T07:58:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.668+0000 7f34aa800640 1 Processor -- start 2024-04-17T07:58:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.668+0000 7f34aa800640 1 -- start start 2024-04-17T07:58:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34aa800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4102850 0x7f34a419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34aa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4105180 0x7f34a419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34aa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a4107ab0 0x7f34a41a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34aa800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34a410da50 con 0x7f34a4105180 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34aa800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34a410d8d0 con 0x7f34a4102850 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34aa800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34a410dbd0 con 0x7f34a4107ab0 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34a8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4105180 0x7f34a419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34a9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4102850 0x7f34a419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34a9400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4102850 0x7f34a419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46756/0 (socket says 172.21.15.46:46756) 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34a9400640 1 -- 172.21.15.46:0/2267652317 learned_addr learned my addr 172.21.15.46:0/2267652317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34a9e00640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a4107ab0 0x7f34a41a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.669+0000 7f34a9400640 1 -- 172.21.15.46:0/2267652317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a4107ab0 msgr2=0x7f34a41a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a9400640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a4107ab0 0x7f34a41a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a9400640 1 -- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4105180 msgr2=0x7f34a419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a9400640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4105180 0x7f34a419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a9400640 1 -- 172.21.15.46:0/2267652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a41a1f00 con 0x7f34a4102850 2024-04-17T07:58:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a9400640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4102850 0x7f34a419a2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f349800e9f0 tx=0x7f349800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a2000640 1 -- 172.21.15.46:0/2267652317 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800cd80 con 0x7f34a4102850 2024-04-17T07:58:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a41a21f0 con 0x7f34a4102850 2024-04-17T07:58:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a2000640 1 -- 172.21.15.46:0/2267652317 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498010910 con 0x7f34a4102850 2024-04-17T07:58:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34a2000640 1 -- 172.21.15.46:0/2267652317 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980188f0 con 0x7f34a4102850 2024-04-17T07:58:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.670+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41a2790 con 0x7f34a4102850 2024-04-17T07:58:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.671+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a41095a0 con 0x7f34a4102850 2024-04-17T07:58:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.674+0000 7f34a2000640 1 -- 172.21.15.46:0/2267652317 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3498010ac0 con 0x7f34a4102850 2024-04-17T07:58:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.675+0000 7f34a2000640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f348c0a3c30 0x7f348c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.675+0000 7f34a2000640 1 -- 172.21.15.46:0/2267652317 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f34980d0b10 con 0x7f34a4102850 2024-04-17T07:58:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.675+0000 7f34a8a00640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f348c0a3c30 0x7f348c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:25.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.676+0000 7f34a8a00640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f348c0a3c30 0x7f348c0a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f34a419b7f0 tx=0x7f3490005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.679+0000 7f34a2000640 1 -- 172.21.15.46:0/2267652317 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f349809cab0 con 0x7f34a4102850 2024-04-17T07:58:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.889+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f34a4062b60 con 0x7f34a4102850 2024-04-17T07:58:25.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.890+0000 7f34a2000640 1 -- 172.21.15.46:0/2267652317 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f34980a0db0 con 0x7f34a4102850 2024-04-17T07:58:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:25.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.892+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f348c0a3c30 msgr2=0x7f348c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.892+0000 7f34aa800640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f348c0a3c30 0x7f348c0a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f34a419b7f0 tx=0x7f3490005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.892+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4102850 msgr2=0x7f34a419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.892+0000 7f34aa800640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4102850 0x7f34a419a2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f349800e9f0 tx=0x7f349800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 shutdown_connections 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f348c0a3c30 0x7f348c0a60f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34a4107ab0 0x7f34a41a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34a4105180 0x7f34a419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 --2- 172.21.15.46:0/2267652317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a4102850 0x7f34a419a2e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 >> 172.21.15.46:0/2267652317 conn(0x7f34a40fc240 msgr2=0x7f34a4101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 shutdown_connections 2024-04-17T07:58:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:25.893+0000 7f34aa800640 1 -- 172.21.15.46:0/2267652317 wait complete. 2024-04-17T07:58:25.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:58:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 -- 172.21.15.46:0/2176322695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728078500 msgr2=0x7f3728078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 --2- 172.21.15.46:0/2176322695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728078500 0x7f3728078da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3720009ca0 tx=0x7f372002c100 comp rx=0 tx=0).stop 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 -- 172.21.15.46:0/2176322695 shutdown_connections 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 --2- 172.21.15.46:0/2176322695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728078500 0x7f3728078da0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 --2- 172.21.15.46:0/2176322695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728077b60 0x7f3728077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 --2- 172.21.15.46:0/2176322695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3728076920 0x7f3728076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 -- 172.21.15.46:0/2176322695 >> 172.21.15.46:0/2176322695 conn(0x7f37280ff630 msgr2=0x7f3728101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 -- 172.21.15.46:0/2176322695 shutdown_connections 2024-04-17T07:58:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.143+0000 7f3727200640 1 -- 172.21.15.46:0/2176322695 wait complete. 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.144+0000 7f3727200640 1 Processor -- start 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.144+0000 7f3727200640 1 -- start start 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3727200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728076920 0x7f372819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3727200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3728077b60 0x7f372819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3727200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728078500 0x7f37281a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3727200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3728114150 con 0x7f3728078500 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3727200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3728113fd0 con 0x7f3728076920 2024-04-17T07:58:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3727200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37281142d0 con 0x7f3728077b60 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3726800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728078500 0x7f37281a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3726800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728078500 0x7f37281a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57378/0 (socket says 172.21.15.46:57378) 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3726800640 1 -- 172.21.15.46:0/1336177418 learned_addr learned my addr 172.21.15.46:0/1336177418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3725e00640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728076920 0x7f372819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3725400640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3728077b60 0x7f372819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3725e00640 1 -- 172.21.15.46:0/1336177418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3728077b60 msgr2=0x7f372819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3725e00640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3728077b60 0x7f372819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3725e00640 1 -- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728078500 msgr2=0x7f37281a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3725e00640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728078500 0x7f37281a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.145+0000 7f3725e00640 1 -- 172.21.15.46:0/1336177418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281a6360 con 0x7f3728076920 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.146+0000 7f3726800640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728078500 0x7f37281a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.146+0000 7f3725e00640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728076920 0x7f372819e740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f371800e9f0 tx=0x7f371800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.146+0000 7f371ea00640 1 -- 172.21.15.46:0/1336177418 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371800ccc0 con 0x7f3728076920 2024-04-17T07:58:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.146+0000 7f371ea00640 1 -- 172.21.15.46:0/1336177418 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3718005020 con 0x7f3728076920 2024-04-17T07:58:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.146+0000 7f371ea00640 1 -- 172.21.15.46:0/1336177418 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37180049e0 con 0x7f3728076920 2024-04-17T07:58:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.146+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37281a6650 con 0x7f3728076920 2024-04-17T07:58:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.146+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37281a6b50 con 0x7f3728076920 2024-04-17T07:58:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.147+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3728110a30 con 0x7f3728076920 2024-04-17T07:58:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.149+0000 7f371ea00640 1 -- 172.21.15.46:0/1336177418 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3718005560 con 0x7f3728076920 2024-04-17T07:58:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.151+0000 7f371ea00640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f370c0a3ba0 0x7f370c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.151+0000 7f371ea00640 1 -- 172.21.15.46:0/1336177418 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f37180d0770 con 0x7f3728076920 2024-04-17T07:58:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.151+0000 7f3725400640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f370c0a3ba0 0x7f370c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.152+0000 7f3725400640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f370c0a3ba0 0x7f370c0a6060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f372819fc50 tx=0x7f3714005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:26.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.155+0000 7f371ea00640 1 -- 172.21.15.46:0/1336177418 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f37180051d0 con 0x7f3728076920 2024-04-17T07:58:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.409+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f372802d5d0 con 0x7f3728076920 2024-04-17T07:58:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.411+0000 7f371ea00640 1 -- 172.21.15.46:0/1336177418 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f371809cd60 con 0x7f3728076920 2024-04-17T07:58:26.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:26.410 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:54.519826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24556,24542]},"id":1}]} 2024-04-17T07:58:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-04-17T07:58:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f370c0a3ba0 msgr2=0x7f370c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f370c0a3ba0 0x7f370c0a6060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f372819fc50 tx=0x7f3714005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728076920 msgr2=0x7f372819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728076920 0x7f372819e740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f371800e9f0 tx=0x7f371800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 shutdown_connections 2024-04-17T07:58:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f370c0a3ba0 0x7f370c0a6060 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3728078500 0x7f37281a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3728077b60 0x7f372819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 --2- 172.21.15.46:0/1336177418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3728076920 0x7f372819e740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 >> 172.21.15.46:0/1336177418 conn(0x7f37280ff630 msgr2=0x7f3728101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 shutdown_connections 2024-04-17T07:58:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.414+0000 7f3727200640 1 -- 172.21.15.46:0/1336177418 wait complete. 2024-04-17T07:58:26.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:58:26.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:26 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3771478283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:26.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:26 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2267652317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:26.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:26 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3771478283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:26.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:26 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2267652317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.657+0000 7f9188a00640 1 -- 172.21.15.46:0/2999986629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184076920 msgr2=0x7f9184076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.657+0000 7f9188a00640 1 --2- 172.21.15.46:0/2999986629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184076920 0x7f9184076d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9174009900 tx=0x7f917402bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.658+0000 7f9188a00640 1 -- 172.21.15.46:0/2999986629 shutdown_connections 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.658+0000 7f9188a00640 1 --2- 172.21.15.46:0/2999986629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184078500 0x7f9184078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.658+0000 7f9188a00640 1 --2- 172.21.15.46:0/2999986629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9184077b60 0x7f9184077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.658+0000 7f9188a00640 1 --2- 172.21.15.46:0/2999986629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184076920 0x7f9184076d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.658+0000 7f9188a00640 1 -- 172.21.15.46:0/2999986629 >> 172.21.15.46:0/2999986629 conn(0x7f91840ff630 msgr2=0x7f9184101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.658+0000 7f9188a00640 1 -- 172.21.15.46:0/2999986629 shutdown_connections 2024-04-17T07:58:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.658+0000 7f9188a00640 1 -- 172.21.15.46:0/2999986629 wait complete. 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 Processor -- start 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 -- start start 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184076920 0x7f918419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184077b60 0x7f918419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9184078500 0x7f91841a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9184114150 con 0x7f9184077b60 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9184113fd0 con 0x7f9184076920 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.659+0000 7f9188a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91841142d0 con 0x7f9184078500 2024-04-17T07:58:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9182800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184076920 0x7f918419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9179e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184077b60 0x7f918419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9182800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184076920 0x7f918419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46796/0 (socket says 172.21.15.46:46796) 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9182800640 1 -- 172.21.15.46:0/637054154 learned_addr learned my addr 172.21.15.46:0/637054154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9183200640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9184078500 0x7f91841a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9179e00640 1 -- 172.21.15.46:0/637054154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9184078500 msgr2=0x7f91841a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9179e00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9184078500 0x7f91841a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9179e00640 1 -- 172.21.15.46:0/637054154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184076920 msgr2=0x7f918419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9179e00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184076920 0x7f918419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9179e00640 1 -- 172.21.15.46:0/637054154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91841a6480 con 0x7f9184077b60 2024-04-17T07:58:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9182800640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184076920 0x7f918419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.660+0000 7f9179e00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184077b60 0x7f918419eda0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f916c00e9f0 tx=0x7f916c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.661+0000 7f917be00640 1 -- 172.21.15.46:0/637054154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c00ccc0 con 0x7f9184077b60 2024-04-17T07:58:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.661+0000 7f917be00640 1 -- 172.21.15.46:0/637054154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916c005020 con 0x7f9184077b60 2024-04-17T07:58:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.661+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841a6770 con 0x7f9184077b60 2024-04-17T07:58:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.661+0000 7f917be00640 1 -- 172.21.15.46:0/637054154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c004a00 con 0x7f9184077b60 2024-04-17T07:58:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.661+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841a6cc0 con 0x7f9184077b60 2024-04-17T07:58:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.662+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9184110a30 con 0x7f9184077b60 2024-04-17T07:58:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.664+0000 7f917be00640 1 -- 172.21.15.46:0/637054154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f916c0051d0 con 0x7f9184077b60 2024-04-17T07:58:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.667+0000 7f917be00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f91680a3c20 0x7f91680a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.667+0000 7f917be00640 1 -- 172.21.15.46:0/637054154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f916c0d06b0 con 0x7f9184077b60 2024-04-17T07:58:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.667+0000 7f9182800640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f91680a3c20 0x7f91680a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.668+0000 7f9182800640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f91680a3c20 0x7f91680a60e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f917402cfd0 tx=0x7f917402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:26.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.671+0000 7f917be00640 1 -- 172.21.15.46:0/637054154 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f916c09cca0 con 0x7f9184077b60 2024-04-17T07:58:26.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:26 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3771478283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:26.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:26 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2267652317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.882+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9184062b60 con 0x7f9184077b60 2024-04-17T07:58:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.884+0000 7f917be00640 1 -- 172.21.15.46:0/637054154 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7f916c09c5a0 con 0x7f9184077b60 2024-04-17T07:58:26.884 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:26.884 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:58:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f91680a3c20 msgr2=0x7f91680a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f91680a3c20 0x7f91680a60e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f917402cfd0 tx=0x7f917402c880 comp rx=0 tx=0).stop 2024-04-17T07:58:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184077b60 msgr2=0x7f918419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184077b60 0x7f918419eda0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f916c00e9f0 tx=0x7f916c00edf0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 shutdown_connections 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f91680a3c20 0x7f91680a60e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9184078500 0x7f91841a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9184077b60 0x7f918419eda0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 --2- 172.21.15.46:0/637054154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9184076920 0x7f918419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 >> 172.21.15.46:0/637054154 conn(0x7f91840ff630 msgr2=0x7f9184101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.887+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 shutdown_connections 2024-04-17T07:58:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:26.888+0000 7f9188a00640 1 -- 172.21.15.46:0/637054154 wait complete. 2024-04-17T07:58:26.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:58:26.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.140+0000 7f0764200640 1 -- 172.21.15.46:0/514739411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c105d60 msgr2=0x7f075c10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.140+0000 7f0764200640 1 --2- 172.21.15.46:0/514739411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c105d60 0x7f075c10c5f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0754009ca0 tx=0x7f075402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.140+0000 7f0764200640 1 -- 172.21.15.46:0/514739411 shutdown_connections 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.140+0000 7f0764200640 1 --2- 172.21.15.46:0/514739411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c105d60 0x7f075c10c5f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.140+0000 7f0764200640 1 --2- 172.21.15.46:0/514739411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c1053a0 0x7f075c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.140+0000 7f0764200640 1 --2- 172.21.15.46:0/514739411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f075c1041b0 0x7f075c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.140+0000 7f0764200640 1 -- 172.21.15.46:0/514739411 >> 172.21.15.46:0/514739411 conn(0x7f075c0ff6e0 msgr2=0x7f075c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:27.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.141+0000 7f0764200640 1 -- 172.21.15.46:0/514739411 shutdown_connections 2024-04-17T07:58:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.141+0000 7f0764200640 1 -- 172.21.15.46:0/514739411 wait complete. 2024-04-17T07:58:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.142+0000 7f0764200640 1 Processor -- start 2024-04-17T07:58:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.142+0000 7f0764200640 1 -- start start 2024-04-17T07:58:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.142+0000 7f0764200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f075c1041b0 0x7f075c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.142+0000 7f0764200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c1053a0 0x7f075c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.142+0000 7f0764200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c105d60 0x7f075c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0764200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10fe10 con 0x7f075c1053a0 2024-04-17T07:58:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0764200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10fc90 con 0x7f075c1041b0 2024-04-17T07:58:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0764200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10ff90 con 0x7f075c105d60 2024-04-17T07:58:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c105d60 0x7f075c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0760e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c1053a0 0x7f075c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c105d60 0x7f075c1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50242/0 (socket says 172.21.15.46:50242) 2024-04-17T07:58:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 -- 172.21.15.46:0/1443683344 learned_addr learned my addr 172.21.15.46:0/1443683344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0761800640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f075c1041b0 0x7f075c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 -- 172.21.15.46:0/1443683344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f075c1041b0 msgr2=0x7f075c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f075c1041b0 0x7f075c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 -- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c1053a0 msgr2=0x7f075c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c1053a0 0x7f075c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.143+0000 7f0762200640 1 -- 172.21.15.46:0/1443683344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a2040 con 0x7f075c105d60 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.144+0000 7f0762200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c105d60 0x7f075c1a19d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0754037040 tx=0x7f0754032f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.144+0000 7f075a000640 1 -- 172.21.15.46:0/1443683344 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754041020 con 0x7f075c105d60 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.144+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a22d0 con 0x7f075c105d60 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.144+0000 7f075a000640 1 -- 172.21.15.46:0/1443683344 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0754034cd0 con 0x7f075c105d60 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.144+0000 7f075a000640 1 -- 172.21.15.46:0/1443683344 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754039040 con 0x7f075c105d60 2024-04-17T07:58:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.144+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a2810 con 0x7f075c105d60 2024-04-17T07:58:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.145+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c10b4d0 con 0x7f075c105d60 2024-04-17T07:58:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.147+0000 7f075a000640 1 -- 172.21.15.46:0/1443683344 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0754044020 con 0x7f075c105d60 2024-04-17T07:58:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.149+0000 7f075a000640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f07400a3bb0 0x7f07400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.149+0000 7f075a000640 1 -- 172.21.15.46:0/1443683344 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f07540f26a0 con 0x7f075c105d60 2024-04-17T07:58:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.149+0000 7f0761800640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f07400a3bb0 0x7f07400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.150+0000 7f0761800640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f07400a3bb0 0x7f07400a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f074c0095f0 tx=0x7f074c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:27.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.154+0000 7f075a000640 1 -- 172.21.15.46:0/1443683344 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f075404b020 con 0x7f075c105d60 2024-04-17T07:58:27.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.366+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f075c062b60 con 0x7f075c105d60 2024-04-17T07:58:27.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.367+0000 7f075a000640 1 -- 172.21.15.46:0/1443683344 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f07540bed20 con 0x7f075c105d60 2024-04-17T07:58:27.366 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:58:27.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.369+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f07400a3bb0 msgr2=0x7f07400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.369+0000 7f0764200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f07400a3bb0 0x7f07400a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f074c0095f0 tx=0x7f074c005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.369+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c105d60 msgr2=0x7f075c1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.369+0000 7f0764200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c105d60 0x7f075c1a19d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0754037040 tx=0x7f0754032f40 comp rx=0 tx=0).stop 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 shutdown_connections 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f07400a3bb0 0x7f07400a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f075c105d60 0x7f075c1a19d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f075c1053a0 0x7f075c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 --2- 172.21.15.46:0/1443683344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f075c1041b0 0x7f075c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 >> 172.21.15.46:0/1443683344 conn(0x7f075c0ff6e0 msgr2=0x7f075c1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:27.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 shutdown_connections 2024-04-17T07:58:27.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.370+0000 7f0764200640 1 -- 172.21.15.46:0/1443683344 wait complete. 2024-04-17T07:58:27.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:58:27.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:27 smithi046 ceph-mon[32038]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:27.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:27 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1336177418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:27.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:27 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/637054154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:27.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:27 smithi062 ceph-mon[36507]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:27.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:27 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1336177418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:27.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:27 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/637054154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 -- 172.21.15.46:0/3043619761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 msgr2=0x7f9540110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 --2- 172.21.15.46:0/3043619761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 0x7f9540110790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9538009cd0 tx=0x7f953802c0e0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 -- 172.21.15.46:0/3043619761 shutdown_connections 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 --2- 172.21.15.46:0/3043619761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 0x7f9540110790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 --2- 172.21.15.46:0/3043619761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400749c0 0x7f9540074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 --2- 172.21.15.46:0/3043619761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 0x7f95400743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 -- 172.21.15.46:0/3043619761 >> 172.21.15.46:0/3043619761 conn(0x7f95400fd340 msgr2=0x7f95400ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 -- 172.21.15.46:0/3043619761 shutdown_connections 2024-04-17T07:58:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.626+0000 7f953f200640 1 -- 172.21.15.46:0/3043619761 wait complete. 2024-04-17T07:58:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.627+0000 7f953f200640 1 Processor -- start 2024-04-17T07:58:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953f200640 1 -- start start 2024-04-17T07:58:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400749c0 0x7f954019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 0x7f954019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 0x7f95401a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9540114020 con 0x7f9540075fa0 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9540113ea0 con 0x7f954010e3a0 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f95401141a0 con 0x7f95400749c0 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 0x7f954019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400749c0 0x7f954019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 0x7f954019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57440/0 (socket says 172.21.15.46:57440) 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953d400640 1 -- 172.21.15.46:0/1751262606 learned_addr learned my addr 172.21.15.46:0/1751262606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.628+0000 7f953e800640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 0x7f95401a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.629+0000 7f953de00640 1 -- 172.21.15.46:0/1751262606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 msgr2=0x7f95401a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.629+0000 7f953de00640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 0x7f95401a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.629+0000 7f953de00640 1 -- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 msgr2=0x7f954019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.629+0000 7f953de00640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 0x7f954019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.629+0000 7f953de00640 1 -- 172.21.15.46:0/1751262606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401a62d0 con 0x7f95400749c0 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.629+0000 7f953d400640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 0x7f954019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.629+0000 7f953de00640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400749c0 0x7f954019e6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f953000d6d0 tx=0x7f953000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:27.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.630+0000 7f9536a00640 1 -- 172.21.15.46:0/1751262606 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530017070 con 0x7f95400749c0 2024-04-17T07:58:27.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.630+0000 7f9536a00640 1 -- 172.21.15.46:0/1751262606 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530009820 con 0x7f95400749c0 2024-04-17T07:58:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.630+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401a65c0 con 0x7f95400749c0 2024-04-17T07:58:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.630+0000 7f9536a00640 1 -- 172.21.15.46:0/1751262606 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530004d00 con 0x7f95400749c0 2024-04-17T07:58:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.630+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401a6b10 con 0x7f95400749c0 2024-04-17T07:58:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.631+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540104570 con 0x7f95400749c0 2024-04-17T07:58:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.633+0000 7f9536a00640 1 -- 172.21.15.46:0/1751262606 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f95300099d0 con 0x7f95400749c0 2024-04-17T07:58:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.634+0000 7f9536a00640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f95240a3bb0 0x7f95240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.635+0000 7f9536a00640 1 -- 172.21.15.46:0/1751262606 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f95300d0470 con 0x7f95400749c0 2024-04-17T07:58:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.635+0000 7f953d400640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f95240a3bb0 0x7f95240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.635+0000 7f953d400640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f95240a3bb0 0x7f95240a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f954019fbc0 tx=0x7f952c00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.637+0000 7f9536a00640 1 -- 172.21.15.46:0/1751262606 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f953009c410 con 0x7f95400749c0 2024-04-17T07:58:27.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:27 smithi053 ceph-mon[36437]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:27.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:27 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1336177418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:27.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:27 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/637054154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:27.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.847+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f954002d840 con 0x7f95400749c0 2024-04-17T07:58:27.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.848+0000 7f9536a00640 1 -- 172.21.15.46:0/1751262606 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f95300a0710 con 0x7f95400749c0 2024-04-17T07:58:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f95240a3bb0 msgr2=0x7f95240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f95240a3bb0 0x7f95240a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f954019fbc0 tx=0x7f952c00b480 comp rx=0 tx=0).stop 2024-04-17T07:58:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400749c0 msgr2=0x7f954019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400749c0 0x7f954019e6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f953000d6d0 tx=0x7f953000db90 comp rx=0 tx=0).stop 2024-04-17T07:58:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 shutdown_connections 2024-04-17T07:58:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f95240a3bb0 0x7f95240a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f954010e3a0 0x7f95401a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9540075fa0 0x7f954019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 --2- 172.21.15.46:0/1751262606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95400749c0 0x7f954019e6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:27.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 >> 172.21.15.46:0/1751262606 conn(0x7f95400fd340 msgr2=0x7f9540101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:27.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 shutdown_connections 2024-04-17T07:58:27.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:27.851+0000 7f953f200640 1 -- 172.21.15.46:0/1751262606 wait complete. 2024-04-17T07:58:27.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:58:28.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 -- 172.21.15.46:0/552268232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc077b60 msgr2=0x7f83cc077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 --2- 172.21.15.46:0/552268232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc077b60 0x7f83cc077fc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009900 tx=0x7f83b402bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 -- 172.21.15.46:0/552268232 shutdown_connections 2024-04-17T07:58:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 --2- 172.21.15.46:0/552268232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83cc078500 0x7f83cc078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 --2- 172.21.15.46:0/552268232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc077b60 0x7f83cc077fc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 --2- 172.21.15.46:0/552268232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc076920 0x7f83cc076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 -- 172.21.15.46:0/552268232 >> 172.21.15.46:0/552268232 conn(0x7f83cc0ff6e0 msgr2=0x7f83cc101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 -- 172.21.15.46:0/552268232 shutdown_connections 2024-04-17T07:58:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.105+0000 7f83d2000640 1 -- 172.21.15.46:0/552268232 wait complete. 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.106+0000 7f83d2000640 1 Processor -- start 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83d2000640 1 -- start start 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83d2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83cc076920 0x7f83cc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83d2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc077b60 0x7f83cc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83d2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc078500 0x7f83cc1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83d2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc114280 con 0x7f83cc078500 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83d2000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc114100 con 0x7f83cc077b60 2024-04-17T07:58:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83d2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc114400 con 0x7f83cc076920 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83caa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc077b60 0x7f83cc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc078500 0x7f83cc1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83caa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc077b60 0x7f83cc19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55326/0 (socket says 172.21.15.46:55326) 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.107+0000 7f83caa00640 1 -- 172.21.15.46:0/4009847442 learned_addr learned my addr 172.21.15.46:0/4009847442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83cb400640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83cc076920 0x7f83cc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83cbe00640 1 -- 172.21.15.46:0/4009847442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83cc076920 msgr2=0x7f83cc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83cbe00640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83cc076920 0x7f83cc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83cbe00640 1 -- 172.21.15.46:0/4009847442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc077b60 msgr2=0x7f83cc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83cbe00640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc077b60 0x7f83cc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83cbe00640 1 -- 172.21.15.46:0/4009847442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc1a6490 con 0x7f83cc078500 2024-04-17T07:58:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83caa00640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc077b60 0x7f83cc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.108+0000 7f83cbe00640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc078500 0x7f83cc1a5e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f83c400e970 tx=0x7f83c400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.109+0000 7f83c3e00640 1 -- 172.21.15.46:0/4009847442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c400ccc0 con 0x7f83cc078500 2024-04-17T07:58:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.109+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc1a6780 con 0x7f83cc078500 2024-04-17T07:58:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.109+0000 7f83c3e00640 1 -- 172.21.15.46:0/4009847442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c4005020 con 0x7f83cc078500 2024-04-17T07:58:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.109+0000 7f83c3e00640 1 -- 172.21.15.46:0/4009847442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4004a00 con 0x7f83cc078500 2024-04-17T07:58:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.109+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1a6cf0 con 0x7f83cc078500 2024-04-17T07:58:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.110+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc110c60 con 0x7f83cc078500 2024-04-17T07:58:28.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.112+0000 7f83c3e00640 1 -- 172.21.15.46:0/4009847442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f83c40051d0 con 0x7f83cc078500 2024-04-17T07:58:28.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.113+0000 7f83c3e00640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83b00a3c30 0x7f83b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.113+0000 7f83c3e00640 1 -- 172.21.15.46:0/4009847442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f83c40d06c0 con 0x7f83cc078500 2024-04-17T07:58:28.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.114+0000 7f83cb400640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83b00a3c30 0x7f83b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.114+0000 7f83cb400640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83b00a3c30 0x7f83b00a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005fd0 tx=0x7f83bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:28.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.120+0000 7f83c3e00640 1 -- 172.21.15.46:0/4009847442 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f83c409ccb0 con 0x7f83cc078500 2024-04-17T07:58:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.335+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f83cc062b60 con 0x7f83cc078500 2024-04-17T07:58:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.336+0000 7f83c3e00640 1 -- 172.21.15.46:0/4009847442 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f83c409c5b0 con 0x7f83cc078500 2024-04-17T07:58:28.335 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83b00a3c30 msgr2=0x7f83b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83b00a3c30 0x7f83b00a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005fd0 tx=0x7f83bc009500 comp rx=0 tx=0).stop 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc078500 msgr2=0x7f83cc1a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc078500 0x7f83cc1a5e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f83c400e970 tx=0x7f83c400ee30 comp rx=0 tx=0).stop 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 shutdown_connections 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83b00a3c30 0x7f83b00a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83cc078500 0x7f83cc1a5e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc077b60 0x7f83cc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 --2- 172.21.15.46:0/4009847442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83cc076920 0x7f83cc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.338+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 >> 172.21.15.46:0/4009847442 conn(0x7f83cc0ff6e0 msgr2=0x7f83cc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.339+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 shutdown_connections 2024-04-17T07:58:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.339+0000 7f83d2000640 1 -- 172.21.15.46:0/4009847442 wait complete. 2024-04-17T07:58:28.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:58:28.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:28 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1443683344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:28.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:28 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1751262606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 -- 172.21.15.46:0/3048227878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f701051d0 msgr2=0x7f7f70105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 --2- 172.21.15.46:0/3048227878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f701051d0 0x7f7f70105650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009900 tx=0x7f7f5802bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 -- 172.21.15.46:0/3048227878 shutdown_connections 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 --2- 172.21.15.46:0/3048227878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 0x7f7f7010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 --2- 172.21.15.46:0/3048227878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f701051d0 0x7f7f70105650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 --2- 172.21.15.46:0/3048227878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f70103fe0 0x7f7f70104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 -- 172.21.15.46:0/3048227878 >> 172.21.15.46:0/3048227878 conn(0x7f7f700ff570 msgr2=0x7f7f70101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 -- 172.21.15.46:0/3048227878 shutdown_connections 2024-04-17T07:58:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.593+0000 7f7f75400640 1 -- 172.21.15.46:0/3048227878 wait complete. 2024-04-17T07:58:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 Processor -- start 2024-04-17T07:58:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 -- start start 2024-04-17T07:58:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70103fe0 0x7f7f7019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f701051d0 0x7f7f7019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 0x7f7f701a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010fcc0 con 0x7f7f701051d0 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010fb40 con 0x7f7f70103fe0 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.594+0000 7f7f75400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010fe40 con 0x7f7f70105b90 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f67400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f701051d0 0x7f7f7019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f74a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 0x7f7f701a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f74a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 0x7f7f701a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37530/0 (socket says 172.21.15.46:37530) 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f74a00640 1 -- 172.21.15.46:0/3892976765 learned_addr learned my addr 172.21.15.46:0/3892976765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f67400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f701051d0 0x7f7f7019a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38204/0 (socket says 172.21.15.46:38204) 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f6fe00640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70103fe0 0x7f7f7019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f67400640 1 -- 172.21.15.46:0/3892976765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 msgr2=0x7f7f701a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f67400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 0x7f7f701a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f67400640 1 -- 172.21.15.46:0/3892976765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70103fe0 msgr2=0x7f7f7019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f67400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70103fe0 0x7f7f7019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f67400640 1 -- 172.21.15.46:0/3892976765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f701a1f30 con 0x7f7f701051d0 2024-04-17T07:58:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.595+0000 7f7f74a00640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 0x7f7f701a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.596+0000 7f7f67400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f701051d0 0x7f7f7019a850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f58036040 tx=0x7f7f5802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:28.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.596+0000 7f7f6d600640 1 -- 172.21.15.46:0/3892976765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58043020 con 0x7f7f701051d0 2024-04-17T07:58:28.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.596+0000 7f7f6d600640 1 -- 172.21.15.46:0/3892976765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f580054e0 con 0x7f7f701051d0 2024-04-17T07:58:28.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.596+0000 7f7f6d600640 1 -- 172.21.15.46:0/3892976765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58003ca0 con 0x7f7f701051d0 2024-04-17T07:58:28.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.596+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f701a21c0 con 0x7f7f701051d0 2024-04-17T07:58:28.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.596+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701a2730 con 0x7f7f701051d0 2024-04-17T07:58:28.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.597+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7010b3a0 con 0x7f7f701051d0 2024-04-17T07:58:28.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.599+0000 7f7f6d600640 1 -- 172.21.15.46:0/3892976765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7f58046020 con 0x7f7f701051d0 2024-04-17T07:58:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.601+0000 7f7f6d600640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f540a3c30 0x7f7f540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.601+0000 7f7f6d600640 1 -- 172.21.15.46:0/3892976765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7f580f27b0 con 0x7f7f701051d0 2024-04-17T07:58:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.601+0000 7f7f6fe00640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f540a3c30 0x7f7f540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:28.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.602+0000 7f7f6fe00640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f540a3c30 0x7f7f540a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7f70105030 tx=0x7f7f60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.605+0000 7f7f6d600640 1 -- 172.21.15.46:0/3892976765 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7f5804d020 con 0x7f7f701051d0 2024-04-17T07:58:28.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:28 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1443683344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:28.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:28 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1751262606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:28.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:28 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1443683344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:28.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:28 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1751262606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:28.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.813+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7f70062b60 con 0x7f7f701051d0 2024-04-17T07:58:28.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.814+0000 7f7f6d600640 1 -- 172.21.15.46:0/3892976765 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f7f580beda0 con 0x7f7f701051d0 2024-04-17T07:58:28.813 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.816+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f540a3c30 msgr2=0x7f7f540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f540a3c30 0x7f7f540a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7f70105030 tx=0x7f7f60005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f701051d0 msgr2=0x7f7f7019a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f701051d0 0x7f7f7019a850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f58036040 tx=0x7f7f5802c6c0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 shutdown_connections 2024-04-17T07:58:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7f540a3c30 0x7f7f540a60f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f70105b90 0x7f7f701a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f701051d0 0x7f7f7019a850 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 --2- 172.21.15.46:0/3892976765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70103fe0 0x7f7f7019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 >> 172.21.15.46:0/3892976765 conn(0x7f7f700ff570 msgr2=0x7f7f70108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 shutdown_connections 2024-04-17T07:58:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:28.817+0000 7f7f75400640 1 -- 172.21.15.46:0/3892976765 wait complete. 2024-04-17T07:58:28.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:58:29.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 -- 172.21.15.46:0/1651034997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b781039a0 msgr2=0x7f2b7810a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/1651034997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b781039a0 0x7f2b7810a2c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ca0 tx=0x7f2b7002c100 comp rx=0 tx=0).stop 2024-04-17T07:58:29.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 -- 172.21.15.46:0/1651034997 shutdown_connections 2024-04-17T07:58:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/1651034997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b781039a0 0x7f2b7810a2c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/1651034997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78102fe0 0x7f2b78103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/1651034997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b78101e90 0x7f2b781022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 -- 172.21.15.46:0/1651034997 >> 172.21.15.46:0/1651034997 conn(0x7f2b780fd420 msgr2=0x7f2b780ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 -- 172.21.15.46:0/1651034997 shutdown_connections 2024-04-17T07:58:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.069+0000 7f2b7ca00640 1 -- 172.21.15.46:0/1651034997 wait complete. 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 Processor -- start 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 -- start start 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78101e90 0x7f2b7819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78102fe0 0x7f2b7819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b781039a0 0x7f2b781a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2b780747a0 con 0x7f2b781039a0 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78074620 con 0x7f2b78101e90 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.070+0000 7f2b7ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78074920 con 0x7f2b78102fe0 2024-04-17T07:58:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78101e90 0x7f2b7819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b76a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78102fe0 0x7f2b7819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78101e90 0x7f2b7819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55342/0 (socket says 172.21.15.46:55342) 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 -- 172.21.15.46:0/2807636220 learned_addr learned my addr 172.21.15.46:0/2807636220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77e00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b781039a0 0x7f2b781a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 -- 172.21.15.46:0/2807636220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78102fe0 msgr2=0x7f2b7819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78102fe0 0x7f2b7819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 -- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b781039a0 msgr2=0x7f2b781a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b781039a0 0x7f2b781a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b77400640 1 -- 172.21.15.46:0/2807636220 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b781a1f70 con 0x7f2b78101e90 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.071+0000 7f2b76a00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78102fe0 0x7f2b7819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.072+0000 7f2b77400640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78101e90 0x7f2b7819a350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800d950 tx=0x7f2b6800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.072+0000 7f2b6fe00640 1 -- 172.21.15.46:0/2807636220 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68017070 con 0x7f2b78101e90 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.072+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b781a2260 con 0x7f2b78101e90 2024-04-17T07:58:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.072+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b781a2800 con 0x7f2b78101e90 2024-04-17T07:58:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.072+0000 7f2b6fe00640 1 -- 172.21.15.46:0/2807636220 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b68009c30 con 0x7f2b78101e90 2024-04-17T07:58:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.072+0000 7f2b6fe00640 1 -- 172.21.15.46:0/2807636220 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68020060 con 0x7f2b78101e90 2024-04-17T07:58:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.073+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b580052c0 con 0x7f2b78101e90 2024-04-17T07:58:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.076+0000 7f2b6fe00640 1 -- 172.21.15.46:0/2807636220 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2b680109d0 con 0x7f2b78101e90 2024-04-17T07:58:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.077+0000 7f2b6fe00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2b5c0a3ba0 0x7f2b5c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.077+0000 7f2b6fe00640 1 -- 172.21.15.46:0/2807636220 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f2b680cffc0 con 0x7f2b78101e90 2024-04-17T07:58:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.077+0000 7f2b76a00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2b5c0a3ba0 0x7f2b5c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.078+0000 7f2b76a00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2b5c0a3ba0 0x7f2b5c0a6060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b7819b860 tx=0x7f2b64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.079+0000 7f2b6fe00640 1 -- 172.21.15.46:0/2807636220 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2b68009de0 con 0x7f2b78101e90 2024-04-17T07:58:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.318+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b580055b0 con 0x7f2b78101e90 2024-04-17T07:58:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.319+0000 7f2b6fe00640 1 -- 172.21.15.46:0/2807636220 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f2b6809bf60 con 0x7f2b78101e90 2024-04-17T07:58:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:29.319 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:55:54.519826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24556,24542]},"id":1}]} 2024-04-17T07:58:29.320 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-04-17T07:58:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2b5c0a3ba0 msgr2=0x7f2b5c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2b5c0a3ba0 0x7f2b5c0a6060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b7819b860 tx=0x7f2b64005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78101e90 msgr2=0x7f2b7819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78101e90 0x7f2b7819a350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800d950 tx=0x7f2b6800de10 comp rx=0 tx=0).stop 2024-04-17T07:58:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 shutdown_connections 2024-04-17T07:58:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2b5c0a3ba0 0x7f2b5c0a6060 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b781039a0 0x7f2b781a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78102fe0 0x7f2b7819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 --2- 172.21.15.46:0/2807636220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b78101e90 0x7f2b7819a350 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 >> 172.21.15.46:0/2807636220 conn(0x7f2b780fd420 msgr2=0x7f2b780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 shutdown_connections 2024-04-17T07:58:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.322+0000 7f2b7ca00640 1 -- 172.21.15.46:0/2807636220 wait complete. 2024-04-17T07:58:29.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:58:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:29 smithi062 ceph-mon[36507]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:29 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4009847442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:29.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:29 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3892976765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 -- 172.21.15.46:0/3552744806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c81096a0 msgr2=0x7ff2c8109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3552744806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c81096a0 0x7ff2c8109b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009900 tx=0x7ff2b002bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 -- 172.21.15.46:0/3552744806 shutdown_connections 2024-04-17T07:58:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3552744806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c810a060 0x7ff2c81108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3552744806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c81096a0 0x7ff2c8109b20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3552744806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c8108500 0x7ff2c8108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 -- 172.21.15.46:0/3552744806 >> 172.21.15.46:0/3552744806 conn(0x7ff2c8077450 msgr2=0x7ff2c8079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 -- 172.21.15.46:0/3552744806 shutdown_connections 2024-04-17T07:58:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.567+0000 7ff2cca00640 1 -- 172.21.15.46:0/3552744806 wait complete. 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 Processor -- start 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 -- start start 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8108500 0x7ff2c819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c81096a0 0x7ff2c819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c810a060 0x7ff2c81a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8114190 con 0x7ff2c810a060 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8114010 con 0x7ff2c8108500 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.568+0000 7ff2cca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8114310 con 0x7ff2c81096a0 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8108500 0x7ff2c819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8108500 0x7ff2c819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55354/0 (socket says 172.21.15.46:55354) 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 -- 172.21.15.46:0/3275177910 learned_addr learned my addr 172.21.15.46:0/3275177910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:29.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2bea00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c81096a0 0x7ff2c819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7e00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c810a060 0x7ff2c81a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 -- 172.21.15.46:0/3275177910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c81096a0 msgr2=0x7ff2c819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c81096a0 0x7ff2c819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 -- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c810a060 msgr2=0x7ff2c81a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c810a060 0x7ff2c81a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 -- 172.21.15.46:0/3275177910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c81a6380 con 0x7ff2c8108500 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.569+0000 7ff2c7400640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8108500 0x7ff2c819e760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800e9c0 tx=0x7ff2b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.570+0000 7ff2c4c00640 1 -- 172.21.15.46:0/3275177910 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800cd80 con 0x7ff2c8108500 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.570+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c81a6670 con 0x7ff2c8108500 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.570+0000 7ff2c4c00640 1 -- 172.21.15.46:0/3275177910 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b80035f0 con 0x7ff2c8108500 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.570+0000 7ff2c4c00640 1 -- 172.21.15.46:0/3275177910 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8010670 con 0x7ff2c8108500 2024-04-17T07:58:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.570+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c81a6b90 con 0x7ff2c8108500 2024-04-17T07:58:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.571+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c810f770 con 0x7ff2c8108500 2024-04-17T07:58:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.573+0000 7ff2c4c00640 1 -- 172.21.15.46:0/3275177910 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff2b80037a0 con 0x7ff2c8108500 2024-04-17T07:58:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.574+0000 7ff2c4c00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff2a80a3c20 0x7ff2a80a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.575+0000 7ff2c4c00640 1 -- 172.21.15.46:0/3275177910 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ff2b80d0d10 con 0x7ff2c8108500 2024-04-17T07:58:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.575+0000 7ff2bea00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff2a80a3c20 0x7ff2a80a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.575+0000 7ff2bea00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff2a80a3c20 0x7ff2a80a60e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2b002c240 tx=0x7ff2b002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.577+0000 7ff2c4c00640 1 -- 172.21.15.46:0/3275177910 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff2b8053ff0 con 0x7ff2c8108500 2024-04-17T07:58:29.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:29 smithi046 ceph-mon[32038]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:29.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:29 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4009847442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:29.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:29 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3892976765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:29.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:29 smithi053 ceph-mon[36437]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:29.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:29 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4009847442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:29.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:29 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3892976765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.784+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2c802d5d0 con 0x7ff2c8108500 2024-04-17T07:58:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.786+0000 7ff2c4c00640 1 -- 172.21.15.46:0/3275177910 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7ff2b809d300 con 0x7ff2c8108500 2024-04-17T07:58:29.785 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff2a80a3c20 msgr2=0x7ff2a80a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff2a80a3c20 0x7ff2a80a60e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff2b002c240 tx=0x7ff2b002df90 comp rx=0 tx=0).stop 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8108500 msgr2=0x7ff2c819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8108500 0x7ff2c819e760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800e9c0 tx=0x7ff2b800ee80 comp rx=0 tx=0).stop 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 shutdown_connections 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff2a80a3c20 0x7ff2a80a60e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2c810a060 0x7ff2c81a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c81096a0 0x7ff2c819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 --2- 172.21.15.46:0/3275177910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8108500 0x7ff2c819e760 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 >> 172.21.15.46:0/3275177910 conn(0x7ff2c8077450 msgr2=0x7ff2c810c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 shutdown_connections 2024-04-17T07:58:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:29.789+0000 7ff2cca00640 1 -- 172.21.15.46:0/3275177910 wait complete. 2024-04-17T07:58:29.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T07:55:26.016968+0000', 'modified': '2024-04-17T07:55:54.519826+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24596, 'mds_1': 24556, 'mds_2': 24542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6835/2191814520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2191814520}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2191814520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/956269488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 956269488}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 956269488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/665634381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 665634381}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 665634381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.53:6837/2183592848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 2183592848}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 2183592848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24556': {'gid': 24556, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.46:6839/137300158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 137300158}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 137300158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/262156089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 262156089}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 262156089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24596, 24556, 24542]} 2024-04-17T07:58:29.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-17T07:58:29.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 -- 172.21.15.46:0/825742530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c1039a0 msgr2=0x7fa85c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 --2- 172.21.15.46:0/825742530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c1039a0 0x7fa85c10a230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa854009860 tx=0x7fa85402bb70 comp rx=0 tx=0).stop 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 -- 172.21.15.46:0/825742530 shutdown_connections 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 --2- 172.21.15.46:0/825742530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c1039a0 0x7fa85c10a230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 --2- 172.21.15.46:0/825742530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa85c102fe0 0x7fa85c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 --2- 172.21.15.46:0/825742530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa85c101df0 0x7fa85c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 -- 172.21.15.46:0/825742530 >> 172.21.15.46:0/825742530 conn(0x7fa85c0fd360 msgr2=0x7fa85c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 -- 172.21.15.46:0/825742530 shutdown_connections 2024-04-17T07:58:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.035+0000 7fa85b200640 1 -- 172.21.15.46:0/825742530 wait complete. 2024-04-17T07:58:30.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.036+0000 7fa85b200640 1 Processor -- start 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85b200640 1 -- start start 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85b200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c101df0 0x7fa85c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa85c102fe0 0x7fa85c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa85c1039a0 0x7fa85c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10da50 con 0x7fa85c1039a0 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85b200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10d8d0 con 0x7fa85c101df0 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10dbd0 con 0x7fa85c102fe0 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa859e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c101df0 0x7fa85c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa859e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c101df0 0x7fa85c198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55372/0 (socket says 172.21.15.46:55372) 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa859e00640 1 -- 172.21.15.46:0/1974785694 learned_addr learned my addr 172.21.15.46:0/1974785694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa85a800640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa85c1039a0 0x7fa85c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.037+0000 7fa859400640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa85c102fe0 0x7fa85c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa859e00640 1 -- 172.21.15.46:0/1974785694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa85c102fe0 msgr2=0x7fa85c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa859e00640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa85c102fe0 0x7fa85c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa859e00640 1 -- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa85c1039a0 msgr2=0x7fa85c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa859e00640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa85c1039a0 0x7fa85c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa859e00640 1 -- 172.21.15.46:0/1974785694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c19fc90 con 0x7fa85c101df0 2024-04-17T07:58:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa859e00640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c101df0 0x7fa85c198070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e9f0 tx=0x7fa84c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa852a00640 1 -- 172.21.15.46:0/1974785694 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00ccc0 con 0x7fa85c101df0 2024-04-17T07:58:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa852a00640 1 -- 172.21.15.46:0/1974785694 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84c0035f0 con 0x7fa85c101df0 2024-04-17T07:58:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c19ff80 con 0x7fa85c101df0 2024-04-17T07:58:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa852a00640 1 -- 172.21.15.46:0/1974785694 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00ce60 con 0x7fa85c101df0 2024-04-17T07:58:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.038+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c1a03e0 con 0x7fa85c101df0 2024-04-17T07:58:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 Processor -- start 2024-04-17T07:58:30.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 -- start start 2024-04-17T07:58:30.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa852a00640 1 -- 172.21.15.46:0/1974785694 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa84c0037a0 con 0x7fa85c101df0 2024-04-17T07:58:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e60f00 0x558bc4e61320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e618f0 0x558bc4e663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e66a60 0x558bc4e68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558bc4dc3980 con 0x558bc4e66a60 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558bc4d6a220 con 0x558bc4e618f0 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa863e02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558bc4dbf940 con 0x558bc4e60f00 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.042+0000 7fa859e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e618f0 0x558bc4e663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa859400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e66a60 0x558bc4e68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e60f00 0x558bc4e61320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e60f00 0x558bc4e61320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37602/0 (socket says 172.21.15.46:37602) 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 -- 172.21.15.46:0/254848939 learned_addr learned my addr 172.21.15.46:0/254848939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 -- 172.21.15.46:0/254848939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e618f0 msgr2=0x558bc4e663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e618f0 0x558bc4e663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 -- 172.21.15.46:0/254848939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e66a60 msgr2=0x558bc4e68ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e66a60 0x558bc4e68ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 -- 172.21.15.46:0/254848939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc4ad62d0 con 0x558bc4e60f00 2024-04-17T07:58:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa859400640 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e66a60 0x558bc4e68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.043+0000 7fa85a800640 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e60f00 0x558bc4e61320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8540057c0 tx=0x7fa854005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708013f20f3e213 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.044+0000 7fa84b400640 1 -- 172.21.15.46:0/254848939 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854039040 con 0x558bc4e60f00 2024-04-17T07:58:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.044+0000 7fa84b400640 1 -- 172.21.15.46:0/254848939 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa854034db0 con 0x558bc4e60f00 2024-04-17T07:58:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.044+0000 7fa84b400640 1 -- 172.21.15.46:0/254848939 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85402c8b0 con 0x558bc4e60f00 2024-04-17T07:58:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.044+0000 7fa852a00640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3bb0 0x7fa83c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.044+0000 7fa852a00640 1 -- 172.21.15.46:0/1974785694 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa84c0d0a20 con 0x7fa85c101df0 2024-04-17T07:58:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.044+0000 7fa859e00640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3bb0 0x7fa83c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 -- 172.21.15.46:0/254848939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e60f00 msgr2=0x558bc4e61320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e60f00 0x558bc4e61320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa8540057c0 tx=0x7fa854005b20 comp rx=0 tx=0).stop 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 -- 172.21.15.46:0/254848939 shutdown_connections 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e66a60 0x558bc4e68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e618f0 0x558bc4e663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 --2- 172.21.15.46:0/254848939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e60f00 0x558bc4e61320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 -- 172.21.15.46:0/254848939 >> 172.21.15.46:0/254848939 conn(0x558bc4e58060 msgr2=0x558bc4e58460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 -- 172.21.15.46:0/254848939 shutdown_connections 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 -- 172.21.15.46:0/254848939 wait complete. 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa859e00640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3bb0 0x7fa83c0a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0a5050 tx=0x7fa84c0d3f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.045+0000 7fa863e02740 1 -- start start 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 1 Processor -- start 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e72970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e699b0 con 0x558bc4e60f00 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558bc4d7d590 con 0x558bc4e66a60 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e69b30 con 0x558bc4e618f0 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa863e02740 10 client.0 _ll_register_callbacks cb 0x558bc4ad5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa859400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa85a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa859400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37612/0 (socket says 172.21.15.46:37612) 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa85a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38278/0 (socket says 172.21.15.46:38278) 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.046+0000 7fa859400640 1 -- 172.21.15.46:0/1687025170 learned_addr learned my addr 172.21.15.46:0/1687025170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa85a800640 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e72970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa859400640 1 -- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 msgr2=0x558bc4e72970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa859400640 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e72970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa859400640 1 -- 172.21.15.46:0/1687025170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 msgr2=0x558bc4e6b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa859400640 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa859400640 1 -- 172.21.15.46:0/1687025170 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc4ba06b0 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa859400640 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6b930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa844011d10 tx=0x7fa84400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b68052ad6b69770b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa843e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa843e00640 1 -- 172.21.15.46:0/1687025170 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844003970 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa854034db0 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.047+0000 7fa843e00640 1 -- 172.21.15.46:0/1687025170 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa844003b10 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.048+0000 7fa843e00640 1 -- 172.21.15.46:0/1687025170 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa844025050 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.048+0000 7fa863e02740 10 client.25351 fetch_fsmap learned FSMap version 23 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.048+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558bc4e75da0 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.048+0000 7fa843e00640 1 -- 172.21.15.46:0/1687025170 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa854034db0 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.048+0000 7fa843e00640 1 -- 172.21.15.46:0/1687025170 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7fa84401a490 con 0x558bc4e618f0 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.048+0000 7fa863e02740 10 client.25351 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.048+0000 7fa863e02740 10 client.25351 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:58:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.049+0000 7fa863e02740 20 client.25351 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.049+0000 7fa863e02740 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4e78400 0x558bc4e79870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.049+0000 7fa863e02740 4 client.25351 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:58:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.049+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bc4dd13a0 con 0x558bc4e78400 2024-04-17T07:58:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.049+0000 7fa85a800640 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4e78400 0x558bc4e79870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.050+0000 7fa85a800640 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4e78400 0x558bc4e79870 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca582d401ca8348b server_cookie=d13e786273ac9b18 in_seq=0 out_seq=0 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.050+0000 7fa843e00640 10 client.25351 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.050+0000 7fa843e00640 1 -- 172.21.15.46:0/1687025170 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bc4dd13a0 con 0x558bc4e78400 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.050+0000 7fa843e00640 10 client.25351 handle_command_reply: tid=0 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 client.25351 shutdown 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 msgr2=0x558bc4e6b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6b930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa844011d10 tx=0x7fa84400f790 comp rx=0 tx=0).stop 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 shutdown_connections 2024-04-17T07:58:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4e78400 0x558bc4e79870 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e72970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6b930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 --2- 172.21.15.46:0/1687025170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 >> 172.21.15.46:0/1687025170 conn(0x558bc4e58060 msgr2=0x558bc4e573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa843e00640 0 client.25351 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 shutdown_connections 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 1 -- 172.21.15.46:0/1687025170 wait complete. 2024-04-17T07:58:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.051+0000 7fa863e02740 20 client.25351 trim_cache size 0 max 16384 2024-04-17T07:58:30.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 Processor -- start 2024-04-17T07:58:30.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 -- start start 2024-04-17T07:58:30.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e69ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e72930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e77bb0 con 0x558bc4e60f00 2024-04-17T07:58:30.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e78280 con 0x558bc4e66a60 2024-04-17T07:58:30.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.071+0000 7fa863e02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e78950 con 0x558bc4e618f0 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa85a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e69ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa859400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e72930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa85a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e69ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38290/0 (socket says 172.21.15.46:38290) 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa85a800640 1 -- 172.21.15.46:0/3474134749 learned_addr learned my addr 172.21.15.46:0/3474134749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa85a800640 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa859400640 1 -- 172.21.15.46:0/3474134749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 msgr2=0x558bc4e62e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa859400640 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e62e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa859400640 1 -- 172.21.15.46:0/3474134749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 msgr2=0x558bc4e69ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa859400640 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e69ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.072+0000 7fa859400640 1 -- 172.21.15.46:0/3474134749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc4e6c3e0 con 0x558bc4e618f0 2024-04-17T07:58:30.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.073+0000 7fa859400640 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e72930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa84401aac0 tx=0x7fa84401ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cca99b5ff38efcb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.073+0000 7fa84aa00640 1 -- 172.21.15.46:0/3474134749 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84402d070 con 0x558bc4e618f0 2024-04-17T07:58:30.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.073+0000 7fa84aa00640 1 -- 172.21.15.46:0/3474134749 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844003e50 con 0x558bc4e618f0 2024-04-17T07:58:30.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.073+0000 7fa84aa00640 1 -- 172.21.15.46:0/3474134749 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844005120 con 0x558bc4e618f0 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 -- 172.21.15.46:0/3474134749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 msgr2=0x558bc4e72930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e72930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa84401aac0 tx=0x7fa84401ad50 comp rx=0 tx=0).stop 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 -- 172.21.15.46:0/3474134749 shutdown_connections 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e72930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 --2- 172.21.15.46:0/3474134749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e69ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 -- 172.21.15.46:0/3474134749 >> 172.21.15.46:0/3474134749 conn(0x558bc4e58060 msgr2=0x558bc4e58440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 -- 172.21.15.46:0/3474134749 shutdown_connections 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 -- 172.21.15.46:0/3474134749 wait complete. 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 -- start start 2024-04-17T07:58:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.074+0000 7fa863e02740 1 Processor -- start 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa863e02740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa863e02740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa863e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa863e02740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e796f0 con 0x558bc4e60f00 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa863e02740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e79020 con 0x558bc4e66a60 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa863e02740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558bc4e79dc0 con 0x558bc4e618f0 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa863e02740 10 client.0 _ll_register_callbacks cb 0x7fa844009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa85a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa859400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa85a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38296/0 (socket says 172.21.15.46:38296) 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa85a800640 1 -- 172.21.15.46:0/215370972 learned_addr learned my addr 172.21.15.46:0/215370972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.075+0000 7fa85a800640 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa859400640 1 -- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 msgr2=0x558bc4e71ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa859400640 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e71ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa859400640 1 -- 172.21.15.46:0/215370972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 msgr2=0x558bc4e6e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa859400640 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa859400640 1 -- 172.21.15.46:0/215370972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc4e720a0 con 0x558bc4e618f0 2024-04-17T07:58:30.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa859400640 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6ea90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa84401a8d0 tx=0x7fa84401a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc25be27b36e8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa843e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:58:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa843e00640 1 -- 172.21.15.46:0/215370972 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84402d070 con 0x558bc4e618f0 2024-04-17T07:58:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844003e50 con 0x558bc4e618f0 2024-04-17T07:58:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.076+0000 7fa843e00640 1 -- 172.21.15.46:0/215370972 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa844003970 con 0x558bc4e618f0 2024-04-17T07:58:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa843e00640 1 -- 172.21.15.46:0/215370972 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa844044050 con 0x558bc4e618f0 2024-04-17T07:58:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 10 client.25357 fetch_fsmap learned FSMap version 23 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558bc4ede890 con 0x558bc4e618f0 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa843e00640 1 -- 172.21.15.46:0/215370972 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844003e50 con 0x558bc4e618f0 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa843e00640 1 -- 172.21.15.46:0/215370972 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7fa84403bcb0 con 0x558bc4e618f0 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 10 client.25357 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 10 client.25357 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 20 client.25357 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4ee3080 0x558bc4ee1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 4 client.25357 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.077+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "status"}) -- 0x558bc4dcb210 con 0x558bc4ee3080 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.078+0000 7fa85a800640 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4ee3080 0x558bc4ee1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.078+0000 7fa85a800640 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4ee3080 0x558bc4ee1d90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75616552c503cbf server_cookie=96f135e4904fe7ec in_seq=0 out_seq=0 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.078+0000 7fa843e00640 10 client.25357 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa843e00640 1 -- 172.21.15.46:0/215370972 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558bc4dcb210 con 0x558bc4ee3080 2024-04-17T07:58:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa843e00640 10 client.25357 handle_command_reply: tid=0 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 client.25357 shutdown 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 msgr2=0x558bc4e6ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6ea90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa84401a8d0 tx=0x7fa84401a950 comp rx=0 tx=0).stop 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 shutdown_connections 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x558bc4ee3080 0x558bc4ee1d90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bc4e66a60 0x558bc4e71ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bc4e618f0 0x558bc4e6ea90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 --2- 172.21.15.46:0/215370972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bc4e60f00 0x558bc4e6e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa843e00640 0 client.25357 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 >> 172.21.15.46:0/215370972 conn(0x558bc4e58060 msgr2=0x558bc4e62e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 shutdown_connections 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 1 -- 172.21.15.46:0/215370972 wait complete. 2024-04-17T07:58:30.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.079+0000 7fa863e02740 20 client.25357 trim_cache size 0 max 16384 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24542, 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 156.36000668400001, 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 23, 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:58:30.082 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 205.09833105800001 2024-04-17T07:58:30.083 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:58:30.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3bb0 msgr2=0x7fa83c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3bb0 0x7fa83c0a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0a5050 tx=0x7fa84c0d3f90 comp rx=0 tx=0).stop 2024-04-17T07:58:30.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c101df0 msgr2=0x7fa85c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c101df0 0x7fa85c198070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e9f0 tx=0x7fa84c00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 shutdown_connections 2024-04-17T07:58:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3bb0 0x7fa83c0a6070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa85c1039a0 0x7fa85c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa85c102fe0 0x7fa85c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 --2- 172.21.15.46:0/1974785694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c101df0 0x7fa85c198070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 >> 172.21.15.46:0/1974785694 conn(0x7fa85c0fd360 msgr2=0x7fa85c1062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 shutdown_connections 2024-04-17T07:58:30.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.080+0000 7fa85b200640 1 -- 172.21.15.46:0/1974785694 wait complete. 2024-04-17T07:58:30.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-17T07:58:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.328+0000 7f5741c00640 1 -- 172.21.15.46:0/1193644471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 msgr2=0x7f573c10c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.328+0000 7f5741c00640 1 --2- 172.21.15.46:0/1193644471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c10c2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5734009ca0 tx=0x7f573402c180 comp rx=0 tx=0).stop 2024-04-17T07:58:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.329+0000 7f5741c00640 1 -- 172.21.15.46:0/1193644471 shutdown_connections 2024-04-17T07:58:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.329+0000 7f5741c00640 1 --2- 172.21.15.46:0/1193644471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c10c2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.329+0000 7f5741c00640 1 --2- 172.21.15.46:0/1193644471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c074940 0x7f573c074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.329+0000 7f5741c00640 1 --2- 172.21.15.46:0/1193644471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f573c075f20 0x7f573c074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.329+0000 7f5741c00640 1 -- 172.21.15.46:0/1193644471 >> 172.21.15.46:0/1193644471 conn(0x7f573c0fd520 msgr2=0x7f573c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.329+0000 7f5741c00640 1 -- 172.21.15.46:0/1193644471 shutdown_connections 2024-04-17T07:58:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.329+0000 7f5741c00640 1 -- 172.21.15.46:0/1193644471 wait complete. 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 Processor -- start 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 -- start start 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c074940 0x7f573c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f573c075f20 0x7f573c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10fe50 con 0x7f573c074940 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10fcd0 con 0x7f573c075f20 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.330+0000 7f5741c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10ffd0 con 0x7f573c109e60 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c1a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37646/0 (socket says 172.21.15.46:37646) 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 -- 172.21.15.46:0/2879066206 learned_addr learned my addr 172.21.15.46:0/2879066206 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573aa00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c074940 0x7f573c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f5733e00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f573c075f20 0x7f573c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 -- 172.21.15.46:0/2879066206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f573c075f20 msgr2=0x7f573c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f573c075f20 0x7f573c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 -- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c074940 msgr2=0x7f573c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c074940 0x7f573c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 -- 172.21.15.46:0/2879066206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c1a2080 con 0x7f573c109e60 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573aa00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c074940 0x7f573c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f5733e00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f573c075f20 0x7f573c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.331+0000 7f573b400640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c1a1a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f573402c6c0 tx=0x7f57340093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.332+0000 7f5733400640 1 -- 172.21.15.46:0/2879066206 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734039040 con 0x7f573c109e60 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.332+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c1a2310 con 0x7f573c109e60 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.332+0000 7f5733400640 1 -- 172.21.15.46:0/2879066206 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5734044070 con 0x7f573c109e60 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.332+0000 7f5733400640 1 -- 172.21.15.46:0/2879066206 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734004280 con 0x7f573c109e60 2024-04-17T07:58:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.332+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c1a2850 con 0x7f573c109e60 2024-04-17T07:58:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.335+0000 7f5744861740 1 Processor -- start 2024-04-17T07:58:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.335+0000 7f5733400640 1 -- 172.21.15.46:0/2879066206 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f57340044e0 con 0x7f573c109e60 2024-04-17T07:58:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.335+0000 7f5744861740 1 -- start start 2024-04-17T07:58:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5744861740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba68e0 0x5645daba6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5744861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba72d0 0x5645dababdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5744861740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645dabac440 0x5645dabae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5744861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645dab0d980 con 0x5645daba72d0 2024-04-17T07:58:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5744861740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5645daab4220 con 0x5645dabac440 2024-04-17T07:58:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5744861740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5645dab09940 con 0x5645daba68e0 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba72d0 0x5645dababdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5733e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645dabac440 0x5645dabae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba72d0 0x5645dababdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38314/0 (socket says 172.21.15.46:38314) 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 -- 172.21.15.46:0/3583915010 learned_addr learned my addr 172.21.15.46:0/3583915010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5733e00640 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba68e0 0x5645daba6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 -- 172.21.15.46:0/3583915010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba68e0 msgr2=0x5645daba6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba68e0 0x5645daba6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 -- 172.21.15.46:0/3583915010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645dabac440 msgr2=0x5645dabae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645dabac440 0x5645dabae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f573aa00640 1 -- 172.21.15.46:0/3583915010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645da8202d0 con 0x5645daba72d0 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5733e00640 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645dabac440 0x5645dabae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.336+0000 7f5733400640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f57200a3c30 0x7f57200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.337+0000 7f5733400640 1 -- 172.21.15.46:0/2879066206 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f57340f2d50 con 0x7f573c109e60 2024-04-17T07:58:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.337+0000 7f573aa00640 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba72d0 0x5645dababdd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d40 tx=0x7f572c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed652f0f667aab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.337+0000 7f5733e00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f57200a3c30 0x7f57200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.337+0000 7f572be00640 1 -- 172.21.15.46:0/3583915010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c012650 con 0x5645daba72d0 2024-04-17T07:58:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.337+0000 7f572be00640 1 -- 172.21.15.46:0/3583915010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c009e30 con 0x5645daba72d0 2024-04-17T07:58:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.337+0000 7f572be00640 1 -- 172.21.15.46:0/3583915010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c00fc30 con 0x5645daba72d0 2024-04-17T07:58:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 -- 172.21.15.46:0/3583915010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba72d0 msgr2=0x5645dababdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba72d0 0x5645dababdd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d40 tx=0x7f572c0117b0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5733e00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f57200a3c30 0x7f57200a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5645daba7b30 tx=0x7f5724009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 -- 172.21.15.46:0/3583915010 shutdown_connections 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645dabac440 0x5645dabae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba72d0 0x5645dababdd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 --2- 172.21.15.46:0/3583915010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba68e0 0x5645daba6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 -- 172.21.15.46:0/3583915010 >> 172.21.15.46:0/3583915010 conn(0x5645daba1e00 msgr2=0x5645daba2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 -- 172.21.15.46:0/3583915010 shutdown_connections 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.338+0000 7f5744861740 1 -- 172.21.15.46:0/3583915010 wait complete. 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 -- start start 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 Processor -- start 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645dabaf3f0 con 0x5645daba68e0 2024-04-17T07:58:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5645daac7590 con 0x5645daba72d0 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.339+0000 7f5744861740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5645dabaf570 con 0x5645dabac440 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f5744861740 10 client.0 _ll_register_callbacks cb 0x5645da81fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38320/0 (socket says 172.21.15.46:38320) 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 -- 172.21.15.46:0/1205658476 learned_addr learned my addr 172.21.15.46:0/1205658476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573b400640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f5733e00640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 -- 172.21.15.46:0/1205658476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 msgr2=0x5645dabb8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 -- 172.21.15.46:0/1205658476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 msgr2=0x5645dabb1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573aa00640 1 -- 172.21.15.46:0/1205658476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645da8ea6b0 con 0x5645daba68e0 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.340+0000 7f573b400640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f573aa00640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb0e00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d10 tx=0x7f572c00f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3efd6991e2fc0a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f5728c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f5728c00640 1 -- 172.21.15.46:0/1205658476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c028020 con 0x5645daba68e0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f5728c00640 1 -- 172.21.15.46:0/1205658476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f572c005450 con 0x5645daba68e0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f572c009e30 con 0x5645daba68e0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f5728c00640 1 -- 172.21.15.46:0/1205658476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f572c033050 con 0x5645daba68e0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f5744861740 10 client.15252 fetch_fsmap learned FSMap version 23 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.341+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5645dabbb740 con 0x5645daba68e0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5728c00640 1 -- 172.21.15.46:0/1205658476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c009e30 con 0x5645daba68e0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5728c00640 1 -- 172.21.15.46:0/1205658476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7f572c012550 con 0x5645daba68e0 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5744861740 10 client.15252 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5744861740 10 client.15252 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5744861740 20 client.15252 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5744861740 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dabbdaa0 0x5645dabbef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5744861740 4 client.15252 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:58:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645dab1b3a0 con 0x5645dabbdaa0 2024-04-17T07:58:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.342+0000 7f5733e00640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dabbdaa0 0x5645dabbef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.343+0000 7f5733e00640 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dabbdaa0 0x5645dabbef10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7081822dcb0ee6f2 server_cookie=da97c1c0d8073580 in_seq=0 out_seq=0 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.343+0000 7f5728c00640 10 client.15252 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5728c00640 1 -- 172.21.15.46:0/1205658476 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645dab1b3a0 con 0x5645dabbdaa0 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5728c00640 10 client.15252 handle_command_reply: tid=0 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 client.15252 shutdown 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 msgr2=0x5645dabb0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb0e00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d10 tx=0x7f572c00f5f0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 shutdown_connections 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dabbdaa0 0x5645dabbef10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 --2- 172.21.15.46:0/1205658476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb0e00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5728c00640 0 client.15252 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:58:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.344+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 >> 172.21.15.46:0/1205658476 conn(0x5645daba1e00 msgr2=0x5645daba1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.345+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 shutdown_connections 2024-04-17T07:58:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.345+0000 7f5744861740 1 -- 172.21.15.46:0/1205658476 wait complete. 2024-04-17T07:58:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.345+0000 7f5744861740 20 client.15252 trim_cache size 0 max 16384 2024-04-17T07:58:30.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:30 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2807636220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:30.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:30 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3275177910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 Processor -- start 2024-04-17T07:58:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 -- start start 2024-04-17T07:58:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba68e0 0x5645dabae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba72d0 0x5645dabb8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645dabac440 0x5645daba57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645dabbd250 con 0x5645dabac440 2024-04-17T07:58:30.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5645dabbd920 con 0x5645daba68e0 2024-04-17T07:58:30.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.366+0000 7f5744861740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5645dabbdff0 con 0x5645daba72d0 2024-04-17T07:58:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba72d0 0x5645dabb8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f573aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba68e0 0x5645dabae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba72d0 0x5645dabb8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37680/0 (socket says 172.21.15.46:37680) 2024-04-17T07:58:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 -- 172.21.15.46:0/1700196015 learned_addr learned my addr 172.21.15.46:0/1700196015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f573b400640 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645dabac440 0x5645daba57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 -- 172.21.15.46:0/1700196015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba68e0 msgr2=0x5645dabae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba68e0 0x5645dabae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 -- 172.21.15.46:0/1700196015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645dabac440 msgr2=0x5645daba57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645dabac440 0x5645daba57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f5733e00640 1 -- 172.21.15.46:0/1700196015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645dabac260 con 0x5645daba72d0 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.367+0000 7f573aa00640 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba68e0 0x5645dabae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.368+0000 7f5733e00640 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba72d0 0x5645dabb8340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5645dabb1b00 tx=0x7f5724004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b77b51492724b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.368+0000 7f572b400640 1 -- 172.21.15.46:0/1700196015 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724010920 con 0x5645daba72d0 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.368+0000 7f572b400640 1 -- 172.21.15.46:0/1700196015 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5724024890 con 0x5645daba72d0 2024-04-17T07:58:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.368+0000 7f572b400640 1 -- 172.21.15.46:0/1700196015 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572402c3c0 con 0x5645daba72d0 2024-04-17T07:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 -- 172.21.15.46:0/1700196015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba72d0 msgr2=0x5645dabb8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba72d0 0x5645dabb8340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5645dabb1b00 tx=0x7f5724004730 comp rx=0 tx=0).stop 2024-04-17T07:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 -- 172.21.15.46:0/1700196015 shutdown_connections 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645dabac440 0x5645daba57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645daba72d0 0x5645dabb8340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 --2- 172.21.15.46:0/1700196015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba68e0 0x5645dabae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 -- 172.21.15.46:0/1700196015 >> 172.21.15.46:0/1700196015 conn(0x5645daba1e00 msgr2=0x5645daba50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 -- 172.21.15.46:0/1700196015 shutdown_connections 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.369+0000 7f5744861740 1 -- 172.21.15.46:0/1700196015 wait complete. 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 -- start start 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 Processor -- start 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645dabbed90 con 0x5645daba68e0 2024-04-17T07:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5645dabbe6c0 con 0x5645daba72d0 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.370+0000 7f5744861740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5645dabbf460 con 0x5645dabac440 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5744861740 10 client.0 _ll_register_callbacks cb 0x7f5724009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f573b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55490/0 (socket says 172.21.15.46:55490) 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 -- 172.21.15.46:0/2571774679 learned_addr learned my addr 172.21.15.46:0/2571774679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f573aa00640 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 -- 172.21.15.46:0/2571774679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 msgr2=0x5645dabb7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 -- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 msgr2=0x5645dabb3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f5733e00640 1 -- 172.21.15.46:0/2571774679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645dabb7950 con 0x5645daba72d0 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.371+0000 7f573b400640 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.372+0000 7f5733e00640 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb4340 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f572401c040 tx=0x7f5724010a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d2311253e2741c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.372+0000 7f5728c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.372+0000 7f5728c00640 1 -- 172.21.15.46:0/2571774679 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572403c060 con 0x5645daba72d0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.372+0000 7f5728c00640 1 -- 172.21.15.46:0/2571774679 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f5724025ad0 con 0x5645daba72d0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.372+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5724024890 con 0x5645daba72d0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5728c00640 1 -- 172.21.15.46:0/2571774679 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5724025cd0 con 0x5645daba72d0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 10 client.25388 fetch_fsmap learned FSMap version 23 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5645dac23db0 con 0x5645daba72d0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5728c00640 1 -- 172.21.15.46:0/2571774679 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5724024890 con 0x5645daba72d0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5728c00640 1 -- 172.21.15.46:0/2571774679 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7f57240378a0 con 0x5645daba72d0 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 10 client.25388 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 10 client.25388 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 20 client.25388 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dac28650 0x5645dac27360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 4 client.25388 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.373+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "status"}) -- 0x5645dab15210 con 0x5645dac28650 2024-04-17T07:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.374+0000 7f573aa00640 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dac28650 0x5645dac27360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f573aa00640 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dac28650 0x5645dac27360 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89ab5bb57b78bbb9 server_cookie=aa89c64c4c4ef216 in_seq=0 out_seq=0 2024-04-17T07:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5728c00640 10 client.25388 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5728c00640 1 -- 172.21.15.46:0/2571774679 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5645dab15210 con 0x5645dac28650 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5728c00640 10 client.25388 handle_command_reply: tid=0 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 client.25388 shutdown 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 msgr2=0x5645dabb4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb4340 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f572401c040 tx=0x7f5724010a30 comp rx=0 tx=0).stop 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 shutdown_connections 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5645dabac440 0x5645dabb7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645daba72d0 0x5645dabb4340 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5645dac28650 0x5645dac27360 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 --2- 172.21.15.46:0/2571774679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645daba68e0 0x5645dabb3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 >> 172.21.15.46:0/2571774679 conn(0x5645daba1e00 msgr2=0x5645daba5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.375+0000 7f5728c00640 0 client.25388 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.376+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 shutdown_connections 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.376+0000 7f5744861740 1 -- 172.21.15.46:0/2571774679 wait complete. 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.376+0000 7f5744861740 20 client.25388 trim_cache size 0 max 16384 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24556, 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-17T07:58:30.379 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-17T07:58:30.380 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-17T07:58:30.380 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 157.66936197000001, 2024-04-17T07:58:30.380 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 23, 2024-04-17T07:58:30.380 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-04-17T07:58:30.380 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:58:30.380 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 197.723891667 2024-04-17T07:58:30.380 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f57200a3c30 msgr2=0x7f57200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f57200a3c30 0x7f57200a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5645daba7b30 tx=0x7f5724009680 comp rx=0 tx=0).stop 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 msgr2=0x7f573c1a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c1a1a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f573402c6c0 tx=0x7f57340093d0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 shutdown_connections 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f57200a3c30 0x7f57200a60f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f573c109e60 0x7f573c1a1a10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f573c075f20 0x7f573c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 --2- 172.21.15.46:0/2879066206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f573c074940 0x7f573c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 >> 172.21.15.46:0/2879066206 conn(0x7f573c0fd520 msgr2=0x7f573c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 shutdown_connections 2024-04-17T07:58:30.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.377+0000 7f5741c00640 1 -- 172.21.15.46:0/2879066206 wait complete. 2024-04-17T07:58:30.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T07:58:30.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:30 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2807636220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:30.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:30 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3275177910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 -- 172.21.15.46:0/3399085641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4077b60 msgr2=0x7ff4e4077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3399085641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4077b60 0x7ff4e4077fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009ca0 tx=0x7ff4d002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 -- 172.21.15.46:0/3399085641 shutdown_connections 2024-04-17T07:58:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3399085641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e4078500 0x7ff4e4078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3399085641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4077b60 0x7ff4e4077fc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3399085641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e4076920 0x7ff4e4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 -- 172.21.15.46:0/3399085641 >> 172.21.15.46:0/3399085641 conn(0x7ff4e40ff570 msgr2=0x7ff4e4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 -- 172.21.15.46:0/3399085641 shutdown_connections 2024-04-17T07:58:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.608+0000 7ff4ea600640 1 -- 172.21.15.46:0/3399085641 wait complete. 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 Processor -- start 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 -- start start 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4076920 0x7ff4e419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e4077b60 0x7ff4e419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e4078500 0x7ff4e41a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4114030 con 0x7ff4e4076920 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113eb0 con 0x7ff4e4078500 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.609+0000 7ff4ea600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41141b0 con 0x7ff4e4077b60 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4076920 0x7ff4e419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4076920 0x7ff4e419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38366/0 (socket says 172.21.15.46:38366) 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 -- 172.21.15.46:0/3913794931 learned_addr learned my addr 172.21.15.46:0/3913794931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e4077b60 0x7ff4e419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e4078500 0x7ff4e41a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 -- 172.21.15.46:0/3913794931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e4077b60 msgr2=0x7ff4e419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e4077b60 0x7ff4e419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 -- 172.21.15.46:0/3913794931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e4078500 msgr2=0x7ff4e41a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e4078500 0x7ff4e41a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e9200640 1 -- 172.21.15.46:0/3913794931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41a6330 con 0x7ff4e4076920 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.610+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e4077b60 0x7ff4e419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.611+0000 7ff4e9200640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4076920 0x7ff4e419e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9f0 tx=0x7ff4d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.611+0000 7ff4e1600640 1 -- 172.21.15.46:0/3913794931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400cd80 con 0x7ff4e4076920 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.611+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41a6620 con 0x7ff4e4076920 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.611+0000 7ff4e1600640 1 -- 172.21.15.46:0/3913794931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4d4010910 con 0x7ff4e4076920 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.611+0000 7ff4e1600640 1 -- 172.21.15.46:0/3913794931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40188f0 con 0x7ff4e4076920 2024-04-17T07:58:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.611+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41a6b40 con 0x7ff4e4076920 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.614+0000 7ff4ee998740 1 Processor -- start 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.614+0000 7ff4ee998740 1 -- start start 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.614+0000 7ff4e1600640 1 -- 172.21.15.46:0/3913794931 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff4d4010ac0 con 0x7ff4e4076920 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead2af0 0x5578aead2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead34e0 0x5578aead7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeadaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4ee998740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5578aea39980 con 0x5578aead2af0 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4ee998740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5578ae9e0220 con 0x5578aead34e0 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4ee998740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5578aea35940 con 0x5578aead8650 2024-04-17T07:58:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead2af0 0x5578aead2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead2af0 0x5578aead2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38378/0 (socket says 172.21.15.46:38378) 2024-04-17T07:58:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 -- 172.21.15.46:0/568000875 learned_addr learned my addr 172.21.15.46:0/568000875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeadaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9200640 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead34e0 0x5578aead7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 -- 172.21.15.46:0/568000875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 msgr2=0x5578aeadaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeadaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 -- 172.21.15.46:0/568000875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead34e0 msgr2=0x5578aead7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead34e0 0x5578aead7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.615+0000 7ff4e9c00640 1 -- 172.21.15.46:0/568000875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ae74c2d0 con 0x5578aead2af0 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.616+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead2af0 0x5578aead2f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009ca0 tx=0x7ff4dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91768e4437144c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.616+0000 7ff4da000640 1 -- 172.21.15.46:0/568000875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc01e070 con 0x5578aead2af0 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.616+0000 7ff4da000640 1 -- 172.21.15.46:0/568000875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc009e50 con 0x5578aead2af0 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.616+0000 7ff4da000640 1 -- 172.21.15.46:0/568000875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00eac0 con 0x5578aead2af0 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.616+0000 7ff4e1600640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4c80a3bb0 0x7ff4c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4e1600640 1 -- 172.21.15.46:0/3913794931 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ff4d40d0b80 con 0x7ff4e4076920 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4e9200640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4c80a3bb0 0x7ff4c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 -- 172.21.15.46:0/568000875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead2af0 msgr2=0x5578aead2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead2af0 0x5578aead2f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009ca0 tx=0x7ff4dc011810 comp rx=0 tx=0).stop 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 -- 172.21.15.46:0/568000875 shutdown_connections 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeadaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead34e0 0x5578aead7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 --2- 172.21.15.46:0/568000875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead2af0 0x5578aead2f10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 -- 172.21.15.46:0/568000875 >> 172.21.15.46:0/568000875 conn(0x5578aeace060 msgr2=0x5578aeace460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 -- 172.21.15.46:0/568000875 shutdown_connections 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4ee998740 1 -- 172.21.15.46:0/568000875 wait complete. 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.617+0000 7ff4e9200640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4c80a3bb0 0x7ff4c80a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff4d409cab0 tx=0x7ff4d40d45c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 -- start start 2024-04-17T07:58:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 Processor -- start 2024-04-17T07:58:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeadd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead34e0 0x5578aeadd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead8650 0x5578aeae4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5578aeadb600 con 0x5578aead8650 2024-04-17T07:58:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5578ae9f3590 con 0x5578aead2af0 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5578aeadb780 con 0x5578aead34e0 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.618+0000 7ff4ee998740 10 client.0 _ll_register_callbacks cb 0x5578ae74bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead8650 0x5578aeae4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead8650 0x5578aeae4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38388/0 (socket says 172.21.15.46:38388) 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 -- 172.21.15.46:0/345756124 learned_addr learned my addr 172.21.15.46:0/345756124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead34e0 0x5578aeadd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeadd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 -- 172.21.15.46:0/345756124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead34e0 msgr2=0x5578aeadd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead34e0 0x5578aeadd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 -- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 msgr2=0x5578aeadd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeadd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 -- 172.21.15.46:0/345756124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ae8166b0 con 0x5578aead8650 2024-04-17T07:58:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead34e0 0x5578aeadd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.619+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead8650 0x5578aeae4600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc005760 tx=0x7ff4dc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd4e1905af986d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4cea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4cea00640 1 -- 172.21.15.46:0/345756124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc02e060 con 0x5578aead8650 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4cea00640 1 -- 172.21.15.46:0/345756124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ff4dc004030 con 0x5578aead8650 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4dc009e50 con 0x5578aead8650 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4cea00640 1 -- 172.21.15.46:0/345756124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff4dc035050 con 0x5578aead8650 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4ee998740 10 client.15270 fetch_fsmap learned FSMap version 23 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5578aeae7970 con 0x5578aead8650 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.620+0000 7ff4cea00640 1 -- 172.21.15.46:0/345756124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc009e50 con 0x5578aead8650 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4cea00640 1 -- 172.21.15.46:0/345756124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7ff4dc0046c0 con 0x5578aead8650 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4ee998740 10 client.15270 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4ee998740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4ee998740 20 client.15270 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4ee998740 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeae9d40 0x5578aeaeb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4ee998740 4 client.15270 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:58:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578aea473a0 con 0x5578aeae9d40 2024-04-17T07:58:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.621+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeae9d40 0x5578aeaeb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.622+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeae9d40 0x5578aeaeb1b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce473d1314dbc492 server_cookie=39d2d3a5193f9cf4 in_seq=0 out_seq=0 2024-04-17T07:58:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.622+0000 7ff4cea00640 10 client.15270 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4cea00640 1 -- 172.21.15.46:0/345756124 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578aea473a0 con 0x5578aeae9d40 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4cea00640 10 client.15270 handle_command_reply: tid=0 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 client.15270 shutdown 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead8650 msgr2=0x5578aeae4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead8650 0x5578aeae4600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc005760 tx=0x7ff4dc005850 comp rx=0 tx=0).stop 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 shutdown_connections 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeae9d40 0x5578aeaeb1b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead8650 0x5578aeae4600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead34e0 0x5578aeadd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 --2- 172.21.15.46:0/345756124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeadd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4cea00640 0 client.15270 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 >> 172.21.15.46:0/345756124 conn(0x5578aeace060 msgr2=0x5578aeacd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 shutdown_connections 2024-04-17T07:58:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 1 -- 172.21.15.46:0/345756124 wait complete. 2024-04-17T07:58:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.623+0000 7ff4ee998740 20 client.15270 trim_cache size 0 max 16384 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 Processor -- start 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 -- start start 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeac9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeae4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeace440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5578aeae94f0 con 0x5578aead34e0 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5578aeae9bc0 con 0x5578aead2af0 2024-04-17T07:58:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4ee998740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5578aeaea290 con 0x5578aead8650 2024-04-17T07:58:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeac9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeac9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55528/0 (socket says 172.21.15.46:55528) 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4e9c00640 1 -- 172.21.15.46:0/3154765545 learned_addr learned my addr 172.21.15.46:0/3154765545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.644+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeae4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeace440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4e9c00640 1 -- 172.21.15.46:0/3154765545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 msgr2=0x5578aeace440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeace440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4e9c00640 1 -- 172.21.15.46:0/3154765545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 msgr2=0x5578aeae4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeae4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4e9c00640 1 -- 172.21.15.46:0/3154765545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578aead7cf0 con 0x5578aead2af0 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeac9230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc004b10 tx=0x7ff4dc012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=741562b07fb3286 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.645+0000 7ff4d9600640 1 -- 172.21.15.46:0/3154765545 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc03f070 con 0x5578aead2af0 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4d9600640 1 -- 172.21.15.46:0/3154765545 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc0050b0 con 0x5578aead2af0 2024-04-17T07:58:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4d9600640 1 -- 172.21.15.46:0/3154765545 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc02e050 con 0x5578aead2af0 2024-04-17T07:58:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 -- 172.21.15.46:0/3154765545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 msgr2=0x5578aeac9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeac9230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc004b10 tx=0x7ff4dc012f90 comp rx=0 tx=0).stop 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 -- 172.21.15.46:0/3154765545 shutdown_connections 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeace440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeae4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3154765545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeac9230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 -- 172.21.15.46:0/3154765545 >> 172.21.15.46:0/3154765545 conn(0x5578aeace060 msgr2=0x5578aeadbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 -- 172.21.15.46:0/3154765545 shutdown_connections 2024-04-17T07:58:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.646+0000 7ff4ee998740 1 -- 172.21.15.46:0/3154765545 wait complete. 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 -- start start 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 Processor -- start 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeae2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeb4bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeb4e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5578aeaeb030 con 0x5578aead34e0 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5578aeaea960 con 0x5578aead2af0 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5578aeaeb700 con 0x5578aead8650 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.647+0000 7ff4ee998740 10 client.0 _ll_register_callbacks cb 0x7ff4dc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeb4bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeae2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeae2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:55542/0 (socket says 172.21.15.46:55542) 2024-04-17T07:58:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e9c00640 1 -- 172.21.15.46:0/3319757748 learned_addr learned my addr 172.21.15.46:0/3319757748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeb4e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e3e00640 1 -- 172.21.15.46:0/3319757748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 msgr2=0x5578aeb4e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeb4e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e3e00640 1 -- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 msgr2=0x5578aeae2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeae2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e3e00640 1 -- 172.21.15.46:0/3319757748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578aeb4e760 con 0x5578aead34e0 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeae2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4e3e00640 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeb4bcd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00324c0 tx=0x7ff4d0032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac429b63e0ff38e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.648+0000 7ff4cea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4cea00640 1 -- 172.21.15.46:0/3319757748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0039040 con 0x5578aead34e0 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4cea00640 1 -- 172.21.15.46:0/3319757748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ff4d0049bc0 con 0x5578aead34e0 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4dc0050b0 con 0x5578aead34e0 2024-04-17T07:58:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4cea00640 1 -- 172.21.15.46:0/3319757748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff4d0058050 con 0x5578aead34e0 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4ee998740 10 client.15276 fetch_fsmap learned FSMap version 23 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5578aeb4fa70 con 0x5578aead34e0 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4cea00640 1 -- 172.21.15.46:0/3319757748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc0050b0 con 0x5578aead34e0 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.649+0000 7ff4cea00640 1 -- 172.21.15.46:0/3319757748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 23) ==== 8347+0+0 (secure 0 0 0) 0x7ff4d00347f0 con 0x5578aead34e0 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.650+0000 7ff4ee998740 10 client.15276 fetch_fsmap finished waiting for FSMap version 23 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.650+0000 7ff4ee998740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.650+0000 7ff4ee998740 20 client.15276 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.650+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeb546b0 0x5578aeb533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.650+0000 7ff4ee998740 4 client.15276 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:58:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.650+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "status"}) -- 0x5578aea41210 con 0x5578aeb546b0 2024-04-17T07:58:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.650+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeb546b0 0x5578aeb533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4e9c00640 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeb546b0 0x5578aeb533c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7fa48ab018bf075 server_cookie=519fde5ce7013835 in_seq=0 out_seq=0 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4cea00640 10 client.15276 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4cea00640 1 -- 172.21.15.46:0/3319757748 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5578aea41210 con 0x5578aeb546b0 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4cea00640 10 client.15276 handle_command_reply: tid=0 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 client.15276 shutdown 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 msgr2=0x5578aeb4bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeb4bcd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00324c0 tx=0x7ff4d0032590 comp rx=0 tx=0).stop 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 shutdown_connections 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5578aeb546b0 0x5578aeb533c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578aead8650 0x5578aeb4e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5578aead34e0 0x5578aeb4bcd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 --2- 172.21.15.46:0/3319757748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578aead2af0 0x5578aeae2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4cea00640 0 client.15276 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 >> 172.21.15.46:0/3319757748 conn(0x5578aeace060 msgr2=0x5578aeac9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 shutdown_connections 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 1 -- 172.21.15.46:0/3319757748 wait complete. 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.651+0000 7ff4ee998740 20 client.15276 trim_cache size 0 max 16384 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "67cd6c06-fc8e-11ee-bc91-c7b262605968", 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24596, 2024-04-17T07:58:30.655 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-17T07:58:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-17T07:58:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-17T07:58:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 184.109108045, 2024-04-17T07:58:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 23, 2024-04-17T07:58:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-04-17T07:58:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:58:30.656 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 190.61301039200001 2024-04-17T07:58:30.657 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4c80a3bb0 msgr2=0x7ff4c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4c80a3bb0 0x7ff4c80a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff4d409cab0 tx=0x7ff4d40d45c0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4076920 msgr2=0x7ff4e419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4076920 0x7ff4e419e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400e9f0 tx=0x7ff4d400eeb0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 shutdown_connections 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4c80a3bb0 0x7ff4c80a6070 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e4078500 0x7ff4e41a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4e4077b60 0x7ff4e419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 --2- 172.21.15.46:0/3913794931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e4076920 0x7ff4e419e760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.652+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 >> 172.21.15.46:0/3913794931 conn(0x7ff4e40ff570 msgr2=0x7ff4e4101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.653+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 shutdown_connections 2024-04-17T07:58:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:30.653+0000 7ff4ea600640 1 -- 172.21.15.46:0/3913794931 wait complete. 2024-04-17T07:58:30.673 INFO:teuthology.run_tasks:Running task sequential... 2024-04-17T07:58:30.683 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-17T07:58:30.683 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-17T07:58:30.683 INFO:teuthology.task.print:Enabling mgr modules 2024-04-17T07:58:30.683 INFO:teuthology.run_tasks:Running task kclient... 2024-04-17T07:58:30.698 INFO:tasks.kclient:Mounting kernel clients... 2024-04-17T07:58:30.699 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-17T07:58:30.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:30.699 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-17T07:58:30.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:30 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2807636220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:30.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:30 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3275177910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:30.759 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-17T07:58:30.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:30.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:58:30.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:30.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-17T07:58:30.974 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-17T07:58:30.974 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T07:58:30.974 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:30.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph fs ls 2024-04-17T07:58:31.425 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:58:31.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:31 smithi062 ceph-mon[36507]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:31.582 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:31 smithi046 ceph-mon[32038]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:31.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:31 smithi053 ceph-mon[36437]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:33.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:33 smithi046 ceph-mon[32038]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:33.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:33 smithi062 ceph-mon[36507]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:33.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:33 smithi053 ceph-mon[36437]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:58:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.813+0000 7fd281600700 1 -- 172.21.15.46:0/1559630796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c108a40 msgr2=0x7fd27c108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.813+0000 7fd281600700 1 --2- 172.21.15.46:0/1559630796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c108a40 0x7fd27c108e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009990 tx=0x7fd26c009ca0 comp rx=0 tx=0).stop 2024-04-17T07:58:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.814+0000 7fd281600700 1 -- 172.21.15.46:0/1559630796 shutdown_connections 2024-04-17T07:58:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.814+0000 7fd281600700 1 --2- 172.21.15.46:0/1559630796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 0x7fd27c10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.814+0000 7fd281600700 1 --2- 172.21.15.46:0/1559630796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c108a40 0x7fd27c108e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.814+0000 7fd281600700 1 --2- 172.21.15.46:0/1559630796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c06a760 0x7fd27c06abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.814+0000 7fd281600700 1 -- 172.21.15.46:0/1559630796 >> 172.21.15.46:0/1559630796 conn(0x7fd27c078900 msgr2=0x7fd27c078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.814+0000 7fd281600700 1 -- 172.21.15.46:0/1559630796 shutdown_connections 2024-04-17T07:58:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.814+0000 7fd281600700 1 -- 172.21.15.46:0/1559630796 wait complete. 2024-04-17T07:58:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 Processor -- start 2024-04-17T07:58:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 -- start start 2024-04-17T07:58:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c06a760 0x7fd27c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 0x7fd27c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c108a40 0x7fd27c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c112d30 con 0x7fd27c108a40 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c112bb0 con 0x7fd27c06a760 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.815+0000 7fd281600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c112eb0 con 0x7fd27c06b100 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 0x7fd27c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 0x7fd27c19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:37740/0 (socket says 172.21.15.46:37740) 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 -- 172.21.15.46:0/3399991050 learned_addr learned my addr 172.21.15.46:0/3399991050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd27b400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c108a40 0x7fd27c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd27aa00700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c06a760 0x7fd27c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 -- 172.21.15.46:0/3399991050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c06a760 msgr2=0x7fd27c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c06a760 0x7fd27c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 -- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c108a40 msgr2=0x7fd27c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c108a40 0x7fd27c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd272000700 1 -- 172.21.15.46:0/3399991050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c1a03c0 con 0x7fd27c06b100 2024-04-17T07:58:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.816+0000 7fd27b400700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c108a40 0x7fd27c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.817+0000 7fd272000700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 0x7fd27c19b8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd26400e9a0 tx=0x7fd26400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.817+0000 7fd273e00700 1 -- 172.21.15.46:0/3399991050 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26400ca10 con 0x7fd27c06b100 2024-04-17T07:58:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.817+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c1a06b0 con 0x7fd27c06b100 2024-04-17T07:58:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.817+0000 7fd273e00700 1 -- 172.21.15.46:0/3399991050 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26400cbb0 con 0x7fd27c06b100 2024-04-17T07:58:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.817+0000 7fd273e00700 1 -- 172.21.15.46:0/3399991050 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264018680 con 0x7fd27c06b100 2024-04-17T07:58:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.817+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c1a7f30 con 0x7fd27c06b100 2024-04-17T07:58:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.818+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c069740 con 0x7fd27c06b100 2024-04-17T07:58:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.820+0000 7fd273e00700 1 -- 172.21.15.46:0/3399991050 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd264010430 con 0x7fd27c06b100 2024-04-17T07:58:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.822+0000 7fd273e00700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd2600a3c40 0x7fd2600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.822+0000 7fd273e00700 1 -- 172.21.15.46:0/3399991050 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd2640d09f0 con 0x7fd27c06b100 2024-04-17T07:58:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.822+0000 7fd27aa00700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd2600a3c40 0x7fd2600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.823+0000 7fd27aa00700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd2600a3c40 0x7fd2600a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0098e0 tx=0x7fd26c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:33.826+0000 7fd273e00700 1 -- 172.21.15.46:0/3399991050 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd26409cfe0 con 0x7fd27c06b100 2024-04-17T07:58:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.134+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd27c19c710 con 0x7fd27c06b100 2024-04-17T07:58:34.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.135+0000 7fd273e00700 1 -- 172.21.15.46:0/3399991050 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+88 (secure 0 0 0) 0x7fd26409c8e0 con 0x7fd27c06b100 2024-04-17T07:58:34.134 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T07:58:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd2600a3c40 msgr2=0x7fd2600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd2600a3c40 0x7fd2600a6100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0098e0 tx=0x7fd26c005960 comp rx=0 tx=0).stop 2024-04-17T07:58:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 msgr2=0x7fd27c19b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 0x7fd27c19b8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd26400e9a0 tx=0x7fd26400ecb0 comp rx=0 tx=0).stop 2024-04-17T07:58:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 shutdown_connections 2024-04-17T07:58:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd2600a3c40 0x7fd2600a6100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c108a40 0x7fd27c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:34.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd27c06a760 0x7fd27c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:34.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 --2- 172.21.15.46:0/3399991050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd27c06b100 0x7fd27c19b8f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:34.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.138+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 >> 172.21.15.46:0/3399991050 conn(0x7fd27c078900 msgr2=0x7fd27c10de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:34.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.139+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 shutdown_connections 2024-04-17T07:58:34.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:34.139+0000 7fd281600700 1 -- 172.21.15.46:0/3399991050 wait complete. 2024-04-17T07:58:34.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:34 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3399991050' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:58:34.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:34 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3399991050' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:58:34.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:34 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3399991050' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T07:58:35.101 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T07:58:35.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:35.102 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:58:35.161 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41807sec preferred_lft 41807sec 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:58:35.162 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-17T07:58:35.163 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T07:58:35.163 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:58:35.164 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:58:35.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-04-17T07:58:35.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-04-17T07:58:35.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-04-17T07:58:35.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T07:58:35.164 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:58:35.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:58:35.311 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:35 smithi046 ceph-mon[32038]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:35.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:35.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:35.463 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T07:58:35.537 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-17T07:58:35.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:35.541 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-17T07:58:35.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:35 smithi062 ceph-mon[36507]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:35.599 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-17T07:58:35.599 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T07:58:35.599 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-17T07:58:35.599 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-17T07:58:35.599 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T07:58:35.600 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:58:35.600 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:58:35.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-17T07:58:35.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-17T07:58:35.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-17T07:58:35.600 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:58:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:58:35.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:35 smithi053 ceph-mon[36437]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:58:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:35.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:35.870 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-17T07:58:35.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:35.926 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-17T07:58:35.980 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:58:35.980 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:58:35.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T07:58:35.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-17T07:58:35.980 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:58:36.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:58:36.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:36.200 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-17T07:58:36.200 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:58:36.200 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:58:36.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-17T07:58:36.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T07:58:36.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-17T07:58:36.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-17T07:58:36.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-17T07:58:36.201 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:58:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:58:36.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:36.550 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:58:36.550 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-17T07:58:36.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-17T07:58:36.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T07:58:36.551 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-17T07:58:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:36.788 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:58:36.788 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:36.832 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T07:58:36.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:58:36.833 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:36.932 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-17T07:58:36.932 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-17T07:58:36.932 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/1cdce49c-fba7-4c4b-8cb5-c9dc1266c1ad 2024-04-17T07:58:36.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:36.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/1cdce49c-fba7-4c4b-8cb5-c9dc1266c1ad /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-17T07:58:37.083 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-17T07:58:37.083 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-17T07:58:37.083 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-17T07:58:37.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:37 smithi046 ceph-mon[32038]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:37.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:37 smithi062 ceph-mon[36507]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:37.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:37 smithi053 ceph-mon[36437]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:37.816 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-17T07:58:37.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-17T07:58:37.816 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-17T07:58:37.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.53:6789,172.21.15.62:6789" 2024-04-17T07:58:37.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.53:6789,172.21.15.62:6789:/volumes/qa/sv_1/1cdce49c-fba7-4c4b-8cb5-c9dc1266c1ad 2024-04-17T07:58:37.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=67cd6c06-fc8e-11ee-bc91-c7b262605968" will pass to kernel 2024-04-17T07:58:37.825 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-17T07:58:37.825 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-04-17T07:58:37.825 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T07:58:37.825 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T07:58:37.825 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T07:58:37.831 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T07:58:37.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:37.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:37.928 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T07:58:38.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T07:58:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:38.061 INFO:teuthology.orchestra.run.smithi046.stdout:client25415 2024-04-17T07:58:38.061 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T07:58:38.061 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:58:38.114 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T07:58:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T07:58:38.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:38.239 INFO:teuthology.orchestra.run.smithi046.stdout:client25415 2024-04-17T07:58:38.240 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T07:58:38.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T07:58:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:38.364 INFO:teuthology.orchestra.run.smithi046.stdout:client25415 2024-04-17T07:58:38.365 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968.client25415/status') 2024-04-17T07:58:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:58:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.25415 (3)172.21.15.46:0/2476642904 2024-04-17T07:58:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-04-17T07:58:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-04-17T07:58:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-04-17T07:58:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:71 bytes copied, 6.8517e-05 s, 1.0 MB/s 2024-04-17T07:58:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:58:38.522 INFO:teuthology.run_tasks:Running task exec... 2024-04-17T07:58:38.534 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T07:58:38.535 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-04-17T07:58:38.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 -- 172.21.15.46:0/1746612623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 msgr2=0x7fb8c4074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 --2- 172.21.15.46:0/1746612623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c4074e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009860 tx=0x7fb8ac02bb70 comp rx=0 tx=0).stop 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 -- 172.21.15.46:0/1746612623 shutdown_connections 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 --2- 172.21.15.46:0/1746612623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4109ce0 0x7fb8c410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 --2- 172.21.15.46:0/1746612623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c4074e40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 --2- 172.21.15.46:0/1746612623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8c4075fa0 0x7fb8c40743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 -- 172.21.15.46:0/1746612623 >> 172.21.15.46:0/1746612623 conn(0x7fb8c40fd390 msgr2=0x7fb8c40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 -- 172.21.15.46:0/1746612623 shutdown_connections 2024-04-17T07:58:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.751+0000 7fb8c9800640 1 -- 172.21.15.46:0/1746612623 wait complete. 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 Processor -- start 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 -- start start 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8c4075fa0 0x7fb8c419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4109ce0 0x7fb8c41a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410fca0 con 0x7fb8c4109ce0 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410fb20 con 0x7fb8c4075fa0 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.753+0000 7fb8c9800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410fe20 con 0x7fb8c40749c0 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:38304/0 (socket says 172.21.15.46:38304) 2024-04-17T07:58:38.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 -- 172.21.15.46:0/3189037134 learned_addr learned my addr 172.21.15.46:0/3189037134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c8e00640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4109ce0 0x7fb8c41a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3400640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8c4075fa0 0x7fb8c419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 -- 172.21.15.46:0/3189037134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8c4075fa0 msgr2=0x7fb8c419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8c4075fa0 0x7fb8c419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 -- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4109ce0 msgr2=0x7fb8c41a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4109ce0 0x7fb8c41a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:38.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 -- 172.21.15.46:0/3189037134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c41a1f10 con 0x7fb8c40749c0 2024-04-17T07:58:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.754+0000 7fb8c3e00640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c419a2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400c420 tx=0x7fb8b4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.755+0000 7fb8c0c00640 1 -- 172.21.15.46:0/3189037134 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4003970 con 0x7fb8c40749c0 2024-04-17T07:58:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.755+0000 7fb8c0c00640 1 -- 172.21.15.46:0/3189037134 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b4004020 con 0x7fb8c40749c0 2024-04-17T07:58:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.755+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c41a2200 con 0x7fb8c40749c0 2024-04-17T07:58:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.755+0000 7fb8c0c00640 1 -- 172.21.15.46:0/3189037134 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400ba10 con 0x7fb8c40749c0 2024-04-17T07:58:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.755+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c41a2770 con 0x7fb8c40749c0 2024-04-17T07:58:38.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.756+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c4108a10 con 0x7fb8c40749c0 2024-04-17T07:58:38.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.758+0000 7fb8c0c00640 1 -- 172.21.15.46:0/3189037134 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb8b401d020 con 0x7fb8c40749c0 2024-04-17T07:58:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.759+0000 7fb8c0c00640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb8a80a3c10 0x7fb8a80a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.759+0000 7fb8c0c00640 1 -- 172.21.15.46:0/3189037134 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb8b40d05d0 con 0x7fb8c40749c0 2024-04-17T07:58:38.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.760+0000 7fb8c3400640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb8a80a3c10 0x7fb8a80a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:38.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.760+0000 7fb8c3400640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb8a80a3c10 0x7fb8a80a60d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009830 tx=0x7fb8ac034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:38.764+0000 7fb8c0c00640 1 -- 172.21.15.46:0/3189037134 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb8b409cbc0 con 0x7fb8c40749c0 2024-04-17T07:58:39.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.016+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fb8c402d5d0 con 0x7fb8c40749c0 2024-04-17T07:58:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.296+0000 7fb8c0c00640 1 -- 172.21.15.46:0/3189037134 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7fb8b409c4c0 con 0x7fb8c40749c0 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb8a80a3c10 msgr2=0x7fb8a80a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb8a80a3c10 0x7fb8a80a60d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009830 tx=0x7fb8ac034ab0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 msgr2=0x7fb8c419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c419a2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400c420 tx=0x7fb8b4009f90 comp rx=0 tx=0).stop 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 shutdown_connections 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb8a80a3c10 0x7fb8a80a60d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4109ce0 0x7fb8c41a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8c4075fa0 0x7fb8c419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 --2- 172.21.15.46:0/3189037134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c40749c0 0x7fb8c419a2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.300+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 >> 172.21.15.46:0/3189037134 conn(0x7fb8c40fd390 msgr2=0x7fb8c4105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.301+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 shutdown_connections 2024-04-17T07:58:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.301+0000 7fb8c9800640 1 -- 172.21.15.46:0/3189037134 wait complete. 2024-04-17T07:58:39.322 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-17T07:58:39.333 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-17T07:58:39.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:58:39.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:39 smithi062 ceph-mon[36507]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:39.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:39 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3189037134' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-17T07:58:39.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:39 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-17T07:58:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 -- 172.21.15.46:0/2131920758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 msgr2=0x7fd1901011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2131920758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 0x7fd1901011c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd180009860 tx=0x7fd18002bb70 comp rx=0 tx=0).stop 2024-04-17T07:58:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 -- 172.21.15.46:0/2131920758 shutdown_connections 2024-04-17T07:58:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2131920758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190102950 0x7fd1901091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2131920758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd190101f90 0x7fd190102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2131920758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 0x7fd1901011c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 -- 172.21.15.46:0/2131920758 >> 172.21.15.46:0/2131920758 conn(0x7fd1900fc330 msgr2=0x7fd1900fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 -- 172.21.15.46:0/2131920758 shutdown_connections 2024-04-17T07:58:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.582+0000 7fd18fe00640 1 -- 172.21.15.46:0/2131920758 wait complete. 2024-04-17T07:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.583+0000 7fd18fe00640 1 Processor -- start 2024-04-17T07:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.583+0000 7fd18fe00640 1 -- start start 2024-04-17T07:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 0x7fd19019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190101f90 0x7fd19019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd190102950 0x7fd1901a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd19010db00 con 0x7fd190101f90 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd19010d980 con 0x7fd190102950 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd19010dc80 con 0x7fd190100da0 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190101f90 0x7fd19019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 0x7fd19019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190101f90 0x7fd19019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60482/0 (socket says 172.21.15.46:60482) 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 -- 172.21.15.46:0/2317721221 learned_addr learned my addr 172.21.15.46:0/2317721221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18f400640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd190102950 0x7fd1901a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 -- 172.21.15.46:0/2317721221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 msgr2=0x7fd19019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 0x7fd19019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 -- 172.21.15.46:0/2317721221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd190102950 msgr2=0x7fd1901a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd190102950 0x7fd1901a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.584+0000 7fd18e000640 1 -- 172.21.15.46:0/2317721221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1901a2030 con 0x7fd190101f90 2024-04-17T07:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.585+0000 7fd18e000640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190101f90 0x7fd19019a950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9f0 tx=0x7fd17c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.585+0000 7fd187400640 1 -- 172.21.15.46:0/2317721221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00cd80 con 0x7fd190101f90 2024-04-17T07:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.585+0000 7fd187400640 1 -- 172.21.15.46:0/2317721221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c010910 con 0x7fd190101f90 2024-04-17T07:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.585+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1901a2320 con 0x7fd190101f90 2024-04-17T07:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.585+0000 7fd187400640 1 -- 172.21.15.46:0/2317721221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c018990 con 0x7fd190101f90 2024-04-17T07:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.585+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1901a2870 con 0x7fd190101f90 2024-04-17T07:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.587+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1901a2b90 con 0x7fd190101f90 2024-04-17T07:58:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.590+0000 7fd187400640 1 -- 172.21.15.46:0/2317721221 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd17c010ac0 con 0x7fd190101f90 2024-04-17T07:58:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.591+0000 7fd187400640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1740a3ba0 0x7fd1740a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.591+0000 7fd187400640 1 -- 172.21.15.46:0/2317721221 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd17c0d0b20 con 0x7fd190101f90 2024-04-17T07:58:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.591+0000 7fd18ea00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1740a3ba0 0x7fd1740a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.592+0000 7fd18ea00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1740a3ba0 0x7fd1740a6060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd180009830 tx=0x7fd180034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.596+0000 7fd187400640 1 -- 172.21.15.46:0/2317721221 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd17c09cac0 con 0x7fd190101f90 2024-04-17T07:58:39.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:39 smithi046 ceph-mon[32038]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:39.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:39 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3189037134' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-17T07:58:39.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:39 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-17T07:58:39.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:39 smithi053 ceph-mon[36437]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:58:39.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:39 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3189037134' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-17T07:58:39.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:39 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-17T07:58:39.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.804+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1901a3170 con 0x7fd190101f90 2024-04-17T07:58:39.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.806+0000 7fd187400640 1 -- 172.21.15.46:0/2317721221 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7fd17c0a0dc0 con 0x7fd190101f90 2024-04-17T07:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:39.806 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:58:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.809+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1740a3ba0 msgr2=0x7fd1740a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.809+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1740a3ba0 0x7fd1740a6060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd180009830 tx=0x7fd180034ad0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.809+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190101f90 msgr2=0x7fd19019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190101f90 0x7fd19019a950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9f0 tx=0x7fd17c00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 shutdown_connections 2024-04-17T07:58:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1740a3ba0 0x7fd1740a6060 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd190102950 0x7fd1901a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd190101f90 0x7fd19019a950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 --2- 172.21.15.46:0/2317721221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd190100da0 0x7fd19019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 >> 172.21.15.46:0/2317721221 conn(0x7fd1900fc330 msgr2=0x7fd190109ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 shutdown_connections 2024-04-17T07:58:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:39.810+0000 7fd18fe00640 1 -- 172.21.15.46:0/2317721221 wait complete. 2024-04-17T07:58:39.829 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:58:39.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.058+0000 7fb4bc800640 1 -- 172.21.15.46:0/625177111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4075f40 msgr2=0x7fb4b4074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.058+0000 7fb4bc800640 1 --2- 172.21.15.46:0/625177111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4075f40 0x7fb4b4074390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009900 tx=0x7fb4a802bcd0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.058+0000 7fb4bc800640 1 -- 172.21.15.46:0/625177111 shutdown_connections 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.058+0000 7fb4bc800640 1 --2- 172.21.15.46:0/625177111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b4109e30 0x7fb4b410c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.058+0000 7fb4bc800640 1 --2- 172.21.15.46:0/625177111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b4074960 0x7fb4b4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.058+0000 7fb4bc800640 1 --2- 172.21.15.46:0/625177111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4075f40 0x7fb4b4074390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.058+0000 7fb4bc800640 1 -- 172.21.15.46:0/625177111 >> 172.21.15.46:0/625177111 conn(0x7fb4b40fd4f0 msgr2=0x7fb4b40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.059+0000 7fb4bc800640 1 -- 172.21.15.46:0/625177111 shutdown_connections 2024-04-17T07:58:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.059+0000 7fb4bc800640 1 -- 172.21.15.46:0/625177111 wait complete. 2024-04-17T07:58:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 Processor -- start 2024-04-17T07:58:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 -- start start 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4074960 0x7fb4b419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b4075f40 0x7fb4b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b4109e30 0x7fb4b41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410fe80 con 0x7fb4b4074960 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410fd00 con 0x7fb4b4109e30 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.060+0000 7fb4bc800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4110000 con 0x7fb4b4075f40 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4074960 0x7fb4b419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4074960 0x7fb4b419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60502/0 (socket says 172.21.15.46:60502) 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 -- 172.21.15.46:0/3558283522 learned_addr learned my addr 172.21.15.46:0/3558283522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9400640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b4075f40 0x7fb4b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4ba800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b4109e30 0x7fb4b41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 -- 172.21.15.46:0/3558283522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b4075f40 msgr2=0x7fb4b419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b4075f40 0x7fb4b419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 -- 172.21.15.46:0/3558283522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b4109e30 msgr2=0x7fb4b41a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b4109e30 0x7fb4b41a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4b9e00640 1 -- 172.21.15.46:0/3558283522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b41a20d0 con 0x7fb4b4074960 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.061+0000 7fb4ba800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b4109e30 0x7fb4b41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.062+0000 7fb4b9e00640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4074960 0x7fb4b419a4b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8000c00 tx=0x7fb4a802c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.062+0000 7fb4aea00640 1 -- 172.21.15.46:0/3558283522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8043020 con 0x7fb4b4074960 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.062+0000 7fb4aea00640 1 -- 172.21.15.46:0/3558283522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a80054e0 con 0x7fb4b4074960 2024-04-17T07:58:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.062+0000 7fb4aea00640 1 -- 172.21.15.46:0/3558283522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8003c20 con 0x7fb4b4074960 2024-04-17T07:58:40.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.062+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b41a2360 con 0x7fb4b4074960 2024-04-17T07:58:40.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.062+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b41a2860 con 0x7fb4b4074960 2024-04-17T07:58:40.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.063+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b4108bf0 con 0x7fb4b4074960 2024-04-17T07:58:40.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.065+0000 7fb4aea00640 1 -- 172.21.15.46:0/3558283522 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb4a8046020 con 0x7fb4b4074960 2024-04-17T07:58:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.067+0000 7fb4aea00640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb49c0a3bb0 0x7fb49c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.067+0000 7fb4aea00640 1 -- 172.21.15.46:0/3558283522 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb4a80f28d0 con 0x7fb4b4074960 2024-04-17T07:58:40.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.067+0000 7fb4b9400640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb49c0a3bb0 0x7fb49c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.068+0000 7fb4b9400640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb49c0a3bb0 0x7fb49c0a6070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb4b419b9c0 tx=0x7fb4a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.071+0000 7fb4aea00640 1 -- 172.21.15.46:0/3558283522 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb4a804d020 con 0x7fb4b4074960 2024-04-17T07:58:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.276+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4b4062b60 con 0x7fb4b4074960 2024-04-17T07:58:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.277+0000 7fb4aea00640 1 -- 172.21.15.46:0/3558283522 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb4a80beec0 con 0x7fb4b4074960 2024-04-17T07:58:40.277 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:58:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb49c0a3bb0 msgr2=0x7fb49c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb49c0a3bb0 0x7fb49c0a6070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb4b419b9c0 tx=0x7fb4a4005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4074960 msgr2=0x7fb4b419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4074960 0x7fb4b419a4b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8000c00 tx=0x7fb4a802c6f0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 shutdown_connections 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb49c0a3bb0 0x7fb49c0a6070 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b4109e30 0x7fb4b41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b4075f40 0x7fb4b419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 --2- 172.21.15.46:0/3558283522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b4074960 0x7fb4b419a4b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.280+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 >> 172.21.15.46:0/3558283522 conn(0x7fb4b40fd4f0 msgr2=0x7fb4b4105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.281+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 shutdown_connections 2024-04-17T07:58:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.281+0000 7fb4bc800640 1 -- 172.21.15.46:0/3558283522 wait complete. 2024-04-17T07:58:40.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:58:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 -- 172.21.15.46:0/2240768789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 msgr2=0x7f9fa4103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 --2- 172.21.15.46:0/2240768789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 0x7f9fa4103620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009900 tx=0x7f9f9002bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 -- 172.21.15.46:0/2240768789 shutdown_connections 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 --2- 172.21.15.46:0/2240768789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa4103b60 0x7f9fa410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 --2- 172.21.15.46:0/2240768789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 0x7f9fa4103620 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 --2- 172.21.15.46:0/2240768789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 0x7f9fa41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 -- 172.21.15.46:0/2240768789 >> 172.21.15.46:0/2240768789 conn(0x7f9fa40fd520 msgr2=0x7f9fa40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 -- 172.21.15.46:0/2240768789 shutdown_connections 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.523+0000 7f9faba00640 1 -- 172.21.15.46:0/2240768789 wait complete. 2024-04-17T07:58:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.525+0000 7f9faba00640 1 Processor -- start 2024-04-17T07:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.525+0000 7f9faba00640 1 -- start start 2024-04-17T07:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.525+0000 7f9faba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 0x7f9fa41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.525+0000 7f9faba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 0x7f9fa4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.525+0000 7f9faba00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa4103b60 0x7f9fa419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9faba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa410dc90 con 0x7f9fa4101fb0 2024-04-17T07:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9faba00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa410db10 con 0x7f9fa4103b60 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9faba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa410de10 con 0x7f9fa41031a0 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 0x7f9fa41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 0x7f9fa41982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60518/0 (socket says 172.21.15.46:60518) 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 -- 172.21.15.46:0/3098474447 learned_addr learned my addr 172.21.15.46:0/3098474447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9a00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa4103b60 0x7f9fa419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa3e00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 0x7f9fa4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 -- 172.21.15.46:0/3098474447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 msgr2=0x7f9fa4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 0x7f9fa4198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 -- 172.21.15.46:0/3098474447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa4103b60 msgr2=0x7f9fa419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa4103b60 0x7f9fa419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.526+0000 7f9fa9000640 1 -- 172.21.15.46:0/3098474447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa419ff10 con 0x7f9fa4101fb0 2024-04-17T07:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.527+0000 7f9fa9000640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 0x7f9fa41982f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e9c0 tx=0x7f9f9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.527+0000 7f9fa1600640 1 -- 172.21.15.46:0/3098474447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800cd80 con 0x7f9fa4101fb0 2024-04-17T07:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.527+0000 7f9fa1600640 1 -- 172.21.15.46:0/3098474447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f980035f0 con 0x7f9fa4101fb0 2024-04-17T07:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.527+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa41a0200 con 0x7f9fa4101fb0 2024-04-17T07:58:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.527+0000 7f9fa1600640 1 -- 172.21.15.46:0/3098474447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98010690 con 0x7f9fa4101fb0 2024-04-17T07:58:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.527+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa41a0720 con 0x7f9fa4101fb0 2024-04-17T07:58:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.529+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa4109350 con 0x7f9fa4101fb0 2024-04-17T07:58:40.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.530+0000 7f9fa1600640 1 -- 172.21.15.46:0/3098474447 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9f980037a0 con 0x7f9fa4101fb0 2024-04-17T07:58:40.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.533+0000 7f9fa1600640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9f8c0a3bb0 0x7f9f8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.533+0000 7f9fa1600640 1 -- 172.21.15.46:0/3098474447 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9f980d0bf0 con 0x7f9fa4101fb0 2024-04-17T07:58:40.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.533+0000 7f9fa3e00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9f8c0a3bb0 0x7f9f8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.534+0000 7f9fa3e00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9f8c0a3bb0 0x7f9f8c0a6070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f9002c240 tx=0x7f9f9002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:40.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.537+0000 7f9fa1600640 1 -- 172.21.15.46:0/3098474447 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9f9809d1e0 con 0x7f9fa4101fb0 2024-04-17T07:58:40.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:40 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-17T07:58:40.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:40 smithi046 ceph-mon[32038]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T07:58:40.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:40 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2317721221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:40.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:40 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3558283522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:40.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:40 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-17T07:58:40.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:40 smithi053 ceph-mon[36437]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T07:58:40.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:40 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2317721221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:40.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:40 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3558283522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.744+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9fa4062b60 con 0x7f9fa4101fb0 2024-04-17T07:58:40.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.745+0000 7f9fa1600640 1 -- 172.21.15.46:0/3098474447 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f9f9809cae0 con 0x7f9fa4101fb0 2024-04-17T07:58:40.744 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.747+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9f8c0a3bb0 msgr2=0x7f9f8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.747+0000 7f9faba00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9f8c0a3bb0 0x7f9f8c0a6070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f9002c240 tx=0x7f9f9002df90 comp rx=0 tx=0).stop 2024-04-17T07:58:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 msgr2=0x7f9fa41982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 0x7f9fa41982f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e9c0 tx=0x7f9f9800ee80 comp rx=0 tx=0).stop 2024-04-17T07:58:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 shutdown_connections 2024-04-17T07:58:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9f8c0a3bb0 0x7f9f8c0a6070 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa4103b60 0x7f9fa419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa41031a0 0x7f9fa4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 --2- 172.21.15.46:0/3098474447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa4101fb0 0x7f9fa41982f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 >> 172.21.15.46:0/3098474447 conn(0x7f9fa40fd520 msgr2=0x7f9fa4106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 shutdown_connections 2024-04-17T07:58:40.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.748+0000 7f9faba00640 1 -- 172.21.15.46:0/3098474447 wait complete. 2024-04-17T07:58:40.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:58:40.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:40 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-17T07:58:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:40 smithi062 ceph-mon[36507]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T07:58:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:40 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2317721221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:40.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:40 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3558283522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.988+0000 7f6bc0000640 1 -- 172.21.15.46:0/3609422804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8105370 msgr2=0x7f6bb8107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.988+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3609422804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8105370 0x7f6bb8107760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009900 tx=0x7f6ba402bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.988+0000 7f6bc0000640 1 -- 172.21.15.46:0/3609422804 shutdown_connections 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.988+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3609422804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb8107ca0 0x7f6bb810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.988+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3609422804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8105370 0x7f6bb8107760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.988+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3609422804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb8102a40 0x7f6bb8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.988+0000 7f6bc0000640 1 -- 172.21.15.46:0/3609422804 >> 172.21.15.46:0/3609422804 conn(0x7f6bb80fc410 msgr2=0x7f6bb80fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.989+0000 7f6bc0000640 1 -- 172.21.15.46:0/3609422804 shutdown_connections 2024-04-17T07:58:40.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.989+0000 7f6bc0000640 1 -- 172.21.15.46:0/3609422804 wait complete. 2024-04-17T07:58:40.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.990+0000 7f6bc0000640 1 Processor -- start 2024-04-17T07:58:40.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.990+0000 7f6bc0000640 1 -- start start 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bc0000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8102a40 0x7f6bb819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bc0000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb8105370 0x7f6bb819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bc0000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb8107ca0 0x7f6bb81a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bc0000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb810dca0 con 0x7f6bb8105370 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bc0000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb810db20 con 0x7f6bb8107ca0 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bc0000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb810de20 con 0x7f6bb8102a40 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bbd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8102a40 0x7f6bb819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bbd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8102a40 0x7f6bb819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:38360/0 (socket says 172.21.15.46:38360) 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bbd600640 1 -- 172.21.15.46:0/3623355534 learned_addr learned my addr 172.21.15.46:0/3623355534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bbcc00640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb8105370 0x7f6bb819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.991+0000 7f6bbe000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb8107ca0 0x7f6bb81a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bbd600640 1 -- 172.21.15.46:0/3623355534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb8107ca0 msgr2=0x7f6bb81a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bbd600640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb8107ca0 0x7f6bb81a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bbd600640 1 -- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb8105370 msgr2=0x7f6bb819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bbd600640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb8105370 0x7f6bb819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bbd600640 1 -- 172.21.15.46:0/3623355534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb81a2110 con 0x7f6bb8102a40 2024-04-17T07:58:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bbe000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb8107ca0 0x7f6bb81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bbd600640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8102a40 0x7f6bb819a4f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00ea60 tx=0x7f6bac00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bb6000640 1 -- 172.21.15.46:0/3623355534 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00cde0 con 0x7f6bb8102a40 2024-04-17T07:58:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bb6000640 1 -- 172.21.15.46:0/3623355534 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bac0035f0 con 0x7f6bb8102a40 2024-04-17T07:58:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb81a2400 con 0x7f6bb8102a40 2024-04-17T07:58:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bb6000640 1 -- 172.21.15.46:0/3623355534 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac010670 con 0x7f6bb8102a40 2024-04-17T07:58:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.992+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb81a2970 con 0x7f6bb8102a40 2024-04-17T07:58:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.993+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb81097f0 con 0x7f6bb8102a40 2024-04-17T07:58:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.996+0000 7f6bb6000640 1 -- 172.21.15.46:0/3623355534 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6bac0037a0 con 0x7f6bb8102a40 2024-04-17T07:58:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.997+0000 7f6bb6000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6ba00a3c30 0x7f6ba00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.997+0000 7f6bb6000640 1 -- 172.21.15.46:0/3623355534 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f6bac0d0a50 con 0x7f6bb8102a40 2024-04-17T07:58:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.997+0000 7f6bbcc00640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6ba00a3c30 0x7f6ba00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.998+0000 7f6bbcc00640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6ba00a3c30 0x7f6ba00a60f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ba402c170 tx=0x7f6ba402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:40.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:40.999+0000 7f6bb6000640 1 -- 172.21.15.46:0/3623355534 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6bac09d040 con 0x7f6bb8102a40 2024-04-17T07:58:41.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.206+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6bb8062b60 con 0x7f6bb8102a40 2024-04-17T07:58:41.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.207+0000 7f6bb6000640 1 -- 172.21.15.46:0/3623355534 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f6bac09c940 con 0x7f6bb8102a40 2024-04-17T07:58:41.206 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:58:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.209+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6ba00a3c30 msgr2=0x7f6ba00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.209+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6ba00a3c30 0x7f6ba00a60f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ba402c170 tx=0x7f6ba402df90 comp rx=0 tx=0).stop 2024-04-17T07:58:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.209+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8102a40 msgr2=0x7f6bb819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.209+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8102a40 0x7f6bb819a4f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00ea60 tx=0x7f6bac00ef20 comp rx=0 tx=0).stop 2024-04-17T07:58:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 shutdown_connections 2024-04-17T07:58:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6ba00a3c30 0x7f6ba00a60f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bb8107ca0 0x7f6bb81a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb8105370 0x7f6bb819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 --2- 172.21.15.46:0/3623355534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb8102a40 0x7f6bb819a4f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 >> 172.21.15.46:0/3623355534 conn(0x7f6bb80fc410 msgr2=0x7f6bb8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 shutdown_connections 2024-04-17T07:58:41.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.210+0000 7f6bc0000640 1 -- 172.21.15.46:0/3623355534 wait complete. 2024-04-17T07:58:41.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:58:41.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.460+0000 7f2be8a00640 1 -- 172.21.15.46:0/1327459996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4105190 msgr2=0x7f2be4107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.460+0000 7f2be8a00640 1 --2- 172.21.15.46:0/1327459996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4105190 0x7f2be4107580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009860 tx=0x7f2bd402bb70 comp rx=0 tx=0).stop 2024-04-17T07:58:41.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.461+0000 7f2be8a00640 1 -- 172.21.15.46:0/1327459996 shutdown_connections 2024-04-17T07:58:41.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.461+0000 7f2be8a00640 1 --2- 172.21.15.46:0/1327459996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4107ac0 0x7f2be4109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.461+0000 7f2be8a00640 1 --2- 172.21.15.46:0/1327459996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4105190 0x7f2be4107580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.461+0000 7f2be8a00640 1 --2- 172.21.15.46:0/1327459996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4102860 0x7f2be4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.461+0000 7f2be8a00640 1 -- 172.21.15.46:0/1327459996 >> 172.21.15.46:0/1327459996 conn(0x7f2be40fc250 msgr2=0x7f2be40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:41.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.461+0000 7f2be8a00640 1 -- 172.21.15.46:0/1327459996 shutdown_connections 2024-04-17T07:58:41.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.461+0000 7f2be8a00640 1 -- 172.21.15.46:0/1327459996 wait complete. 2024-04-17T07:58:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.462+0000 7f2be8a00640 1 Processor -- start 2024-04-17T07:58:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.462+0000 7f2be8a00640 1 -- start start 2024-04-17T07:58:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4102860 0x7f2be41021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4105190 0x7f2be4100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be8a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4107ac0 0x7f2be4100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2be410da50 con 0x7f2be4105190 2024-04-17T07:58:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be8a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2be410d8d0 con 0x7f2be4107ac0 2024-04-17T07:58:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be8a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2be410dbd0 con 0x7f2be4102860 2024-04-17T07:58:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4105190 0x7f2be4100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4102860 0x7f2be41021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4105190 0x7f2be4100810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60556/0 (socket says 172.21.15.46:60556) 2024-04-17T07:58:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be2a00640 1 -- 172.21.15.46:0/478872111 learned_addr learned my addr 172.21.15.46:0/478872111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.463+0000 7f2be3e00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4107ac0 0x7f2be4100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be3400640 1 -- 172.21.15.46:0/478872111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4107ac0 msgr2=0x7f2be4100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be3400640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4107ac0 0x7f2be4100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be3400640 1 -- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4105190 msgr2=0x7f2be4100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be3400640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4105190 0x7f2be4100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be3400640 1 -- 172.21.15.46:0/478872111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be41015e0 con 0x7f2be4102860 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be2a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4105190 0x7f2be4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be3400640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4102860 0x7f2be41021d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00e9f0 tx=0x7f2bcc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2bdbe00640 1 -- 172.21.15.46:0/478872111 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00ccc0 con 0x7f2be4102860 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2bdbe00640 1 -- 172.21.15.46:0/478872111 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc010910 con 0x7f2be4102860 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be41a2240 con 0x7f2be4102860 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2bdbe00640 1 -- 172.21.15.46:0/478872111 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc018960 con 0x7f2be4102860 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.464+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be41a2760 con 0x7f2be4102860 2024-04-17T07:58:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.465+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be40fdd40 con 0x7f2be4102860 2024-04-17T07:58:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.467+0000 7f2bdbe00640 1 -- 172.21.15.46:0/478872111 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2bcc010ac0 con 0x7f2be4102860 2024-04-17T07:58:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.469+0000 7f2bdbe00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc80a3bb0 0x7f2bc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.469+0000 7f2bdbe00640 1 -- 172.21.15.46:0/478872111 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f2bcc0d0b30 con 0x7f2be4102860 2024-04-17T07:58:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.469+0000 7f2be2a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc80a3bb0 0x7f2bc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.470+0000 7f2be2a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc80a3bb0 0x7f2bc80a6070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40057c0 tx=0x7f2bd402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:41.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.473+0000 7f2bdbe00640 1 -- 172.21.15.46:0/478872111 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2bcc09cad0 con 0x7f2be4102860 2024-04-17T07:58:41.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:41 smithi046 ceph-mon[32038]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:41 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3098474447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:41.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:41 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3623355534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.678+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2be4062b60 con 0x7f2be4102860 2024-04-17T07:58:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.679+0000 7f2bdbe00640 1 -- 172.21.15.46:0/478872111 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f2bcc0a0dd0 con 0x7f2be4102860 2024-04-17T07:58:41.678 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc80a3bb0 msgr2=0x7f2bc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc80a3bb0 0x7f2bc80a6070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40057c0 tx=0x7f2bd402e040 comp rx=0 tx=0).stop 2024-04-17T07:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4102860 msgr2=0x7f2be41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4102860 0x7f2be41021d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00e9f0 tx=0x7f2bcc00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 shutdown_connections 2024-04-17T07:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f2bc80a3bb0 0x7f2bc80a6070 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2be4107ac0 0x7f2be4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be4105190 0x7f2be4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 --2- 172.21.15.46:0/478872111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2be4102860 0x7f2be41021d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.681+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 >> 172.21.15.46:0/478872111 conn(0x7f2be40fc250 msgr2=0x7f2be4103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.682+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 shutdown_connections 2024-04-17T07:58:41.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.682+0000 7f2be8a00640 1 -- 172.21.15.46:0/478872111 wait complete. 2024-04-17T07:58:41.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:58:41.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:41 smithi053 ceph-mon[36437]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:41.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:41 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3098474447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:41.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:41 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3623355534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:41.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:41 smithi062 ceph-mon[36507]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:58:41.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:41 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3098474447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:41.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:41 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3623355534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.930+0000 7fd75dc00640 1 -- 172.21.15.46:0/3381727029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581085e0 msgr2=0x7fd758108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.930+0000 7fd75dc00640 1 --2- 172.21.15.46:0/3381727029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581085e0 0x7fd758108a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd748009860 tx=0x7fd74802bb70 comp rx=0 tx=0).stop 2024-04-17T07:58:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.931+0000 7fd75dc00640 1 -- 172.21.15.46:0/3381727029 shutdown_connections 2024-04-17T07:58:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.931+0000 7fd75dc00640 1 --2- 172.21.15.46:0/3381727029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75810a190 0x7fd758110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.931+0000 7fd75dc00640 1 --2- 172.21.15.46:0/3381727029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581097d0 0x7fd758109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.931+0000 7fd75dc00640 1 --2- 172.21.15.46:0/3381727029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581085e0 0x7fd758108a00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.931+0000 7fd75dc00640 1 -- 172.21.15.46:0/3381727029 >> 172.21.15.46:0/3381727029 conn(0x7fd7580775b0 msgr2=0x7fd7580799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.931+0000 7fd75dc00640 1 -- 172.21.15.46:0/3381727029 shutdown_connections 2024-04-17T07:58:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.931+0000 7fd75dc00640 1 -- 172.21.15.46:0/3381727029 wait complete. 2024-04-17T07:58:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.932+0000 7fd75dc00640 1 Processor -- start 2024-04-17T07:58:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.932+0000 7fd75dc00640 1 -- start start 2024-04-17T07:58:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd75dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581085e0 0x7fd75819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd75dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581097d0 0x7fd75819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd75dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75810a190 0x7fd7581a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd75dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd758114230 con 0x7fd7581085e0 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd75dc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581140b0 con 0x7fd75810a190 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd75dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7581143b0 con 0x7fd7581097d0 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd756a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581085e0 0x7fd75819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd756a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581085e0 0x7fd75819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60572/0 (socket says 172.21.15.46:60572) 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd756a00640 1 -- 172.21.15.46:0/2793395588 learned_addr learned my addr 172.21.15.46:0/2793395588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd756000640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581097d0 0x7fd75819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.933+0000 7fd757400640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75810a190 0x7fd7581a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd756a00640 1 -- 172.21.15.46:0/2793395588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581097d0 msgr2=0x7fd75819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd756a00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581097d0 0x7fd75819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd756a00640 1 -- 172.21.15.46:0/2793395588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75810a190 msgr2=0x7fd7581a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd756a00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75810a190 0x7fd7581a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd756a00640 1 -- 172.21.15.46:0/2793395588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7581a6510 con 0x7fd7581085e0 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd756000640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581097d0 0x7fd75819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd757400640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75810a190 0x7fd7581a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.934+0000 7fd756a00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581085e0 0x7fd75819e8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd7480349e0 tx=0x7fd748032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.935+0000 7fd74f400640 1 -- 172.21.15.46:0/2793395588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748041020 con 0x7fd7581085e0 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.935+0000 7fd74f400640 1 -- 172.21.15.46:0/2793395588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74802c9f0 con 0x7fd7581085e0 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.935+0000 7fd74f400640 1 -- 172.21.15.46:0/2793395588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748039040 con 0x7fd7581085e0 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.935+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7581a67a0 con 0x7fd7581085e0 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.935+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7581a6ce0 con 0x7fd7581085e0 2024-04-17T07:58:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.936+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75810f9d0 con 0x7fd7581085e0 2024-04-17T07:58:41.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.938+0000 7fd74f400640 1 -- 172.21.15.46:0/2793395588 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd748044020 con 0x7fd7581085e0 2024-04-17T07:58:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.940+0000 7fd74f400640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd73c0a3c30 0x7fd73c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.940+0000 7fd74f400640 1 -- 172.21.15.46:0/2793395588 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd7480f1fa0 con 0x7fd7581085e0 2024-04-17T07:58:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.940+0000 7fd756000640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd73c0a3c30 0x7fd73c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.941+0000 7fd756000640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd73c0a3c30 0x7fd73c0a60f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd7440096c0 tx=0x7fd744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:41.946+0000 7fd74f400640 1 -- 172.21.15.46:0/2793395588 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd74804b020 con 0x7fd7581085e0 2024-04-17T07:58:42.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.157+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd75802d7f0 con 0x7fd7581085e0 2024-04-17T07:58:42.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.159+0000 7fd74f400640 1 -- 172.21.15.46:0/2793395588 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7fd7480be590 con 0x7fd7581085e0 2024-04-17T07:58:42.160 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:42.160 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:58:42.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd73c0a3c30 msgr2=0x7fd73c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd73c0a3c30 0x7fd73c0a60f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd7440096c0 tx=0x7fd744009260 comp rx=0 tx=0).stop 2024-04-17T07:58:42.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581085e0 msgr2=0x7fd75819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581085e0 0x7fd75819e8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd7480349e0 tx=0x7fd748032430 comp rx=0 tx=0).stop 2024-04-17T07:58:42.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 shutdown_connections 2024-04-17T07:58:42.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd73c0a3c30 0x7fd73c0a60f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75810a190 0x7fd7581a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7581097d0 0x7fd75819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 --2- 172.21.15.46:0/2793395588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7581085e0 0x7fd75819e8f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 >> 172.21.15.46:0/2793395588 conn(0x7fd7580775b0 msgr2=0x7fd75810cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:42.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 shutdown_connections 2024-04-17T07:58:42.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.163+0000 7fd75dc00640 1 -- 172.21.15.46:0/2793395588 wait complete. 2024-04-17T07:58:42.183 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:58:42.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:58:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 -- 172.21.15.46:0/2512104786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1085e0 msgr2=0x7f9ebc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 --2- 172.21.15.46:0/2512104786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1085e0 0x7f9ebc108a00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009900 tx=0x7f9eac02bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 -- 172.21.15.46:0/2512104786 shutdown_connections 2024-04-17T07:58:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 --2- 172.21.15.46:0/2512104786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ebc10a190 0x7f9ebc110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 --2- 172.21.15.46:0/2512104786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ebc1097d0 0x7f9ebc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 --2- 172.21.15.46:0/2512104786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1085e0 0x7f9ebc108a00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 -- 172.21.15.46:0/2512104786 >> 172.21.15.46:0/2512104786 conn(0x7f9ebc0775b0 msgr2=0x7f9ebc0799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 -- 172.21.15.46:0/2512104786 shutdown_connections 2024-04-17T07:58:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.398+0000 7f9ec1800640 1 -- 172.21.15.46:0/2512104786 wait complete. 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 Processor -- start 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 -- start start 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ebc1085e0 0x7f9ebc19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1097d0 0x7f9ebc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ebc10a190 0x7f9ebc1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1142b0 con 0x7f9ebc1097d0 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc114130 con 0x7f9ebc10a190 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.399+0000 7f9ec1800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc114430 con 0x7f9ebc1085e0 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9eba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1097d0 0x7f9ebc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9eba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1097d0 0x7f9ebc19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60580/0 (socket says 172.21.15.46:60580) 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9eba000640 1 -- 172.21.15.46:0/1887928387 learned_addr learned my addr 172.21.15.46:0/1887928387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebaa00640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ebc1085e0 0x7f9ebc19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebb400640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ebc10a190 0x7f9ebc1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebaa00640 1 -- 172.21.15.46:0/1887928387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ebc10a190 msgr2=0x7f9ebc1a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebaa00640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ebc10a190 0x7f9ebc1a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebaa00640 1 -- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1097d0 msgr2=0x7f9ebc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebaa00640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1097d0 0x7f9ebc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebaa00640 1 -- 172.21.15.46:0/1887928387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc1a6450 con 0x7f9ebc1085e0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9eba000640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1097d0 0x7f9ebc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.400+0000 7f9ebaa00640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ebc1085e0 0x7f9ebc19e830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009230 tx=0x7f9eac02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.401+0000 7f9eb3400640 1 -- 172.21.15.46:0/1887928387 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac043020 con 0x7f9ebc1085e0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.401+0000 7f9eb3400640 1 -- 172.21.15.46:0/1887928387 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac0054e0 con 0x7f9ebc1085e0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.401+0000 7f9eb3400640 1 -- 172.21.15.46:0/1887928387 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac003c60 con 0x7f9ebc1085e0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.401+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1a66e0 con 0x7f9ebc1085e0 2024-04-17T07:58:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.401+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a6c50 con 0x7f9ebc1085e0 2024-04-17T07:58:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.402+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ebc10f830 con 0x7f9ebc1085e0 2024-04-17T07:58:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.404+0000 7f9eb3400640 1 -- 172.21.15.46:0/1887928387 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9eac046020 con 0x7f9ebc1085e0 2024-04-17T07:58:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.405+0000 7f9eb3400640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9ea00a3c30 0x7f9ea00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.405+0000 7f9eb3400640 1 -- 172.21.15.46:0/1887928387 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9eac0f2760 con 0x7f9ebc1085e0 2024-04-17T07:58:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.405+0000 7f9eba000640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9ea00a3c30 0x7f9ea00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.406+0000 7f9eba000640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9ea00a3c30 0x7f9ea00a60f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc19fd40 tx=0x7f9ea800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:42.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.409+0000 7f9eb3400640 1 -- 172.21.15.46:0/1887928387 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9eac04d020 con 0x7f9ebc1085e0 2024-04-17T07:58:42.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:42 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/478872111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:42.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:42 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2793395588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.622+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ebc062b60 con 0x7f9ebc1085e0 2024-04-17T07:58:42.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.623+0000 7f9eb3400640 1 -- 172.21.15.46:0/1887928387 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f9eac0bedb0 con 0x7f9ebc1085e0 2024-04-17T07:58:42.622 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.625+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9ea00a3c30 msgr2=0x7f9ea00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.625+0000 7f9ec1800640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9ea00a3c30 0x7f9ea00a60f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc19fd40 tx=0x7f9ea800a380 comp rx=0 tx=0).stop 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.625+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ebc1085e0 msgr2=0x7f9ebc19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.625+0000 7f9ec1800640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ebc1085e0 0x7f9ebc19e830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009230 tx=0x7f9eac02c6c0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 shutdown_connections 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9ea00a3c30 0x7f9ea00a60f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ebc10a190 0x7f9ebc1a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ebc1097d0 0x7f9ebc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 --2- 172.21.15.46:0/1887928387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ebc1085e0 0x7f9ebc19e830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 >> 172.21.15.46:0/1887928387 conn(0x7f9ebc0775b0 msgr2=0x7f9ebc10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 shutdown_connections 2024-04-17T07:58:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.626+0000 7f9ec1800640 1 -- 172.21.15.46:0/1887928387 wait complete. 2024-04-17T07:58:42.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:58:42.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:42 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/478872111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:42.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:42 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2793395588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:42.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:42 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/478872111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:42.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:42 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2793395588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.866+0000 7f5dfa200640 1 -- 172.21.15.46:0/1932834912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41085c0 msgr2=0x7f5df41089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.866+0000 7f5dfa200640 1 --2- 172.21.15.46:0/1932834912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41085c0 0x7f5df41089e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009900 tx=0x7f5de402bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.866+0000 7f5dfa200640 1 -- 172.21.15.46:0/1932834912 shutdown_connections 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.866+0000 7f5dfa200640 1 --2- 172.21.15.46:0/1932834912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df410a170 0x7f5df4110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.866+0000 7f5dfa200640 1 --2- 172.21.15.46:0/1932834912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df41097b0 0x7f5df4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.866+0000 7f5dfa200640 1 --2- 172.21.15.46:0/1932834912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41085c0 0x7f5df41089e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.867+0000 7f5dfa200640 1 -- 172.21.15.46:0/1932834912 >> 172.21.15.46:0/1932834912 conn(0x7f5df4077570 msgr2=0x7f5df4079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:42.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.867+0000 7f5dfa200640 1 -- 172.21.15.46:0/1932834912 shutdown_connections 2024-04-17T07:58:42.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.867+0000 7f5dfa200640 1 -- 172.21.15.46:0/1932834912 wait complete. 2024-04-17T07:58:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 Processor -- start 2024-04-17T07:58:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 -- start start 2024-04-17T07:58:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df41085c0 0x7f5df419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41097b0 0x7f5df419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df410a170 0x7f5df41a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4114280 con 0x7f5df41085c0 2024-04-17T07:58:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4114100 con 0x7f5df410a170 2024-04-17T07:58:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5dfa200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4114400 con 0x7f5df41097b0 2024-04-17T07:58:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5df3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df41085c0 0x7f5df419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5df3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df41085c0 0x7f5df419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60592/0 (socket says 172.21.15.46:60592) 2024-04-17T07:58:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5df3400640 1 -- 172.21.15.46:0/2501287442 learned_addr learned my addr 172.21.15.46:0/2501287442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5df3e00640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df410a170 0x7f5df41a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.868+0000 7f5debe00640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41097b0 0x7f5df419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df3400640 1 -- 172.21.15.46:0/2501287442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41097b0 msgr2=0x7f5df419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df3400640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41097b0 0x7f5df419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df3400640 1 -- 172.21.15.46:0/2501287442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df410a170 msgr2=0x7f5df41a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df3400640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df410a170 0x7f5df41a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df3400640 1 -- 172.21.15.46:0/2501287442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df41a6540 con 0x7f5df41085c0 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df3e00640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df410a170 0x7f5df41a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df3400640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df41085c0 0x7f5df419e920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5de402efd0 tx=0x7f5de402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df0c00640 1 -- 172.21.15.46:0/2501287442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4041070 con 0x7f5df41085c0 2024-04-17T07:58:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5df0c00640 1 -- 172.21.15.46:0/2501287442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4004450 con 0x7f5df41085c0 2024-04-17T07:58:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.869+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df41a67d0 con 0x7f5df41085c0 2024-04-17T07:58:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.870+0000 7f5df0c00640 1 -- 172.21.15.46:0/2501287442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4003e50 con 0x7f5df41085c0 2024-04-17T07:58:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.870+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df41a6c90 con 0x7f5df41085c0 2024-04-17T07:58:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.871+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df410f8e0 con 0x7f5df41085c0 2024-04-17T07:58:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.873+0000 7f5df0c00640 1 -- 172.21.15.46:0/2501287442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5de4044070 con 0x7f5df41085c0 2024-04-17T07:58:42.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.874+0000 7f5df0c00640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5dd80a3d50 0x7f5dd80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:42.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.874+0000 7f5df0c00640 1 -- 172.21.15.46:0/2501287442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f5de40f2290 con 0x7f5df41085c0 2024-04-17T07:58:42.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.875+0000 7f5debe00640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5dd80a3d50 0x7f5dd80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:42.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.875+0000 7f5debe00640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5dd80a3d50 0x7f5dd80a6210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5df419fe30 tx=0x7f5ddc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:42.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:42.878+0000 7f5df0c00640 1 -- 172.21.15.46:0/2501287442 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5de40be290 con 0x7f5df41085c0 2024-04-17T07:58:43.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.082+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5df4062b60 con 0x7f5df41085c0 2024-04-17T07:58:43.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.083+0000 7f5df0c00640 1 -- 172.21.15.46:0/2501287442 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f5de40c2590 con 0x7f5df41085c0 2024-04-17T07:58:43.082 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5dd80a3d50 msgr2=0x7f5dd80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5dd80a3d50 0x7f5dd80a6210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5df419fe30 tx=0x7f5ddc005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df41085c0 msgr2=0x7f5df419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df41085c0 0x7f5df419e920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5de402efd0 tx=0x7f5de402c660 comp rx=0 tx=0).stop 2024-04-17T07:58:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 shutdown_connections 2024-04-17T07:58:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5dd80a3d50 0x7f5dd80a6210 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df410a170 0x7f5df41a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5df41097b0 0x7f5df419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 --2- 172.21.15.46:0/2501287442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5df41085c0 0x7f5df419e920 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 >> 172.21.15.46:0/2501287442 conn(0x7f5df4077570 msgr2=0x7f5df410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 shutdown_connections 2024-04-17T07:58:43.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.086+0000 7f5dfa200640 1 -- 172.21.15.46:0/2501287442 wait complete. 2024-04-17T07:58:43.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:58:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.340+0000 7f5ae2800640 1 -- 172.21.15.46:0/3340598401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc101fd0 msgr2=0x7f5adc1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.340+0000 7f5ae2800640 1 --2- 172.21.15.46:0/3340598401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc101fd0 0x7f5adc1023f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009be0 tx=0x7f5acc02bef0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.341+0000 7f5ae2800640 1 -- 172.21.15.46:0/3340598401 shutdown_connections 2024-04-17T07:58:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.341+0000 7f5ae2800640 1 --2- 172.21.15.46:0/3340598401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5adc103b80 0x7f5adc10a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.341+0000 7f5ae2800640 1 --2- 172.21.15.46:0/3340598401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc1031c0 0x7f5adc103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.341+0000 7f5ae2800640 1 --2- 172.21.15.46:0/3340598401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc101fd0 0x7f5adc1023f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.341+0000 7f5ae2800640 1 -- 172.21.15.46:0/3340598401 >> 172.21.15.46:0/3340598401 conn(0x7f5adc0fd540 msgr2=0x7f5adc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.341+0000 7f5ae2800640 1 -- 172.21.15.46:0/3340598401 shutdown_connections 2024-04-17T07:58:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.341+0000 7f5ae2800640 1 -- 172.21.15.46:0/3340598401 wait complete. 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 Processor -- start 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 -- start start 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5adc101fd0 0x7f5adc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc1031c0 0x7f5adc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc103b80 0x7f5adc1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc10dcb0 con 0x7f5adc103b80 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc10db30 con 0x7f5adc101fd0 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.342+0000 7f5ae2800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc10de30 con 0x7f5adc1031c0 2024-04-17T07:58:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc103b80 0x7f5adc1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5adc101fd0 0x7f5adc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5ad2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc1031c0 0x7f5adc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc103b80 0x7f5adc1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60604/0 (socket says 172.21.15.46:60604) 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adbe00640 1 -- 172.21.15.46:0/2291311875 learned_addr learned my addr 172.21.15.46:0/2291311875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adb400640 1 -- 172.21.15.46:0/2291311875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc1031c0 msgr2=0x7f5adc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adb400640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc1031c0 0x7f5adc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adb400640 1 -- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc103b80 msgr2=0x7f5adc1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adb400640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc103b80 0x7f5adc1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adb400640 1 -- 172.21.15.46:0/2291311875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc1a20d0 con 0x7f5adc101fd0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adbe00640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc103b80 0x7f5adc1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.343+0000 7f5adb400640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5adc101fd0 0x7f5adc19a500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5acc02c430 tx=0x7f5acc033830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.344+0000 7f5ad8c00640 1 -- 172.21.15.46:0/2291311875 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc043020 con 0x7f5adc101fd0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.344+0000 7f5ad8c00640 1 -- 172.21.15.46:0/2291311875 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5acc003b80 con 0x7f5adc101fd0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.344+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc1a2360 con 0x7f5adc101fd0 2024-04-17T07:58:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.344+0000 7f5ad8c00640 1 -- 172.21.15.46:0/2291311875 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc034720 con 0x7f5adc101fd0 2024-04-17T07:58:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.344+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc1a2880 con 0x7f5adc101fd0 2024-04-17T07:58:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.345+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc1a2a80 con 0x7f5adc101fd0 2024-04-17T07:58:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.347+0000 7f5ad8c00640 1 -- 172.21.15.46:0/2291311875 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5acc046020 con 0x7f5adc101fd0 2024-04-17T07:58:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.348+0000 7f5ad8c00640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5ac00a3bb0 0x7f5ac00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.348+0000 7f5ad8c00640 1 -- 172.21.15.46:0/2291311875 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f5acc0f2f30 con 0x7f5adc101fd0 2024-04-17T07:58:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.349+0000 7f5ad2a00640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5ac00a3bb0 0x7f5ac00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.349+0000 7f5ad2a00640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5ac00a3bb0 0x7f5ac00a6070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5adc19b9c0 tx=0x7f5ac4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.353+0000 7f5ad8c00640 1 -- 172.21.15.46:0/2291311875 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5acc04d020 con 0x7f5adc101fd0 2024-04-17T07:58:43.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:43 smithi046 ceph-mon[32038]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-17T07:58:43.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:43 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1887928387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:43.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:43 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2501287442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.561+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5adc1a2c50 con 0x7f5adc101fd0 2024-04-17T07:58:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.561+0000 7f5ad8c00640 1 -- 172.21.15.46:0/2291311875 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f5acc003d30 con 0x7f5adc101fd0 2024-04-17T07:58:43.561 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:58:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5ac00a3bb0 msgr2=0x7f5ac00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5ac00a3bb0 0x7f5ac00a6070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5adc19b9c0 tx=0x7f5ac4009260 comp rx=0 tx=0).stop 2024-04-17T07:58:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5adc101fd0 msgr2=0x7f5adc19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5adc101fd0 0x7f5adc19a500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5acc02c430 tx=0x7f5acc033830 comp rx=0 tx=0).stop 2024-04-17T07:58:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 shutdown_connections 2024-04-17T07:58:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5ac00a3bb0 0x7f5ac00a6070 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5adc103b80 0x7f5adc1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5adc1031c0 0x7f5adc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 --2- 172.21.15.46:0/2291311875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5adc101fd0 0x7f5adc19a500 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 >> 172.21.15.46:0/2291311875 conn(0x7f5adc0fd540 msgr2=0x7f5adc074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 shutdown_connections 2024-04-17T07:58:43.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.564+0000 7f5ae2800640 1 -- 172.21.15.46:0/2291311875 wait complete. 2024-04-17T07:58:43.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:58:43.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:43 smithi053 ceph-mon[36437]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-17T07:58:43.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:43 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1887928387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:43.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:43 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2501287442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:43.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:43 smithi062 ceph-mon[36507]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-17T07:58:43.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:43 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1887928387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:43.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:43 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2501287442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.815+0000 7f09ca800640 1 -- 172.21.15.46:0/2390203769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c4109f90 msgr2=0x7f09c4110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.815+0000 7f09ca800640 1 --2- 172.21.15.46:0/2390203769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c4109f90 0x7f09c4110820 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009900 tx=0x7f09bc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.816+0000 7f09ca800640 1 -- 172.21.15.46:0/2390203769 shutdown_connections 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.816+0000 7f09ca800640 1 --2- 172.21.15.46:0/2390203769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c4109f90 0x7f09c4110820 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.816+0000 7f09ca800640 1 --2- 172.21.15.46:0/2390203769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c41095d0 0x7f09c4109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.816+0000 7f09ca800640 1 --2- 172.21.15.46:0/2390203769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 0x7f09c4108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.816+0000 7f09ca800640 1 -- 172.21.15.46:0/2390203769 >> 172.21.15.46:0/2390203769 conn(0x7f09c40773a0 msgr2=0x7f09c40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.816+0000 7f09ca800640 1 -- 172.21.15.46:0/2390203769 shutdown_connections 2024-04-17T07:58:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.816+0000 7f09ca800640 1 -- 172.21.15.46:0/2390203769 wait complete. 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 Processor -- start 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 -- start start 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 0x7f09c419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c41095d0 0x7f09c419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c4109f90 0x7f09c41a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41140c0 con 0x7f09c41083e0 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4113f40 con 0x7f09c4109f90 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09ca800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4114240 con 0x7f09c41095d0 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 0x7f09c419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09c8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c41095d0 0x7f09c419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09c9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 0x7f09c419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60612/0 (socket says 172.21.15.46:60612) 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09c9400640 1 -- 172.21.15.46:0/3024845104 learned_addr learned my addr 172.21.15.46:0/3024845104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.817+0000 7f09c9e00640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c4109f90 0x7f09c41a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c8a00640 1 -- 172.21.15.46:0/3024845104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c4109f90 msgr2=0x7f09c41a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c8a00640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c4109f90 0x7f09c41a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c8a00640 1 -- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 msgr2=0x7f09c419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c8a00640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 0x7f09c419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c8a00640 1 -- 172.21.15.46:0/3024845104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c41a62d0 con 0x7f09c41095d0 2024-04-17T07:58:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c9400640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 0x7f09c419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c8a00640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c41095d0 0x7f09c419ebf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009fd0 tx=0x7f09b800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.818+0000 7f09c2000640 1 -- 172.21.15.46:0/3024845104 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8004b20 con 0x7f09c41095d0 2024-04-17T07:58:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.819+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c41a65c0 con 0x7f09c41095d0 2024-04-17T07:58:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.819+0000 7f09c2000640 1 -- 172.21.15.46:0/3024845104 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b8004cc0 con 0x7f09c41095d0 2024-04-17T07:58:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.819+0000 7f09c2000640 1 -- 172.21.15.46:0/3024845104 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b80107b0 con 0x7f09c41095d0 2024-04-17T07:58:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.819+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c41a6b10 con 0x7f09c41095d0 2024-04-17T07:58:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.820+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c410f6a0 con 0x7f09c41095d0 2024-04-17T07:58:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.822+0000 7f09c2000640 1 -- 172.21.15.46:0/3024845104 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f09b8005020 con 0x7f09c41095d0 2024-04-17T07:58:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.823+0000 7f09c2000640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09ac0a3c30 0x7f09ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.824+0000 7f09c2000640 1 -- 172.21.15.46:0/3024845104 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f09b80d1300 con 0x7f09c41095d0 2024-04-17T07:58:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.824+0000 7f09c9400640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09ac0a3c30 0x7f09ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.824+0000 7f09c9400640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09ac0a3c30 0x7f09ac0a60f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09b40058c0 tx=0x7f09b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:43.826+0000 7f09c2000640 1 -- 172.21.15.46:0/3024845104 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f09b809d8f0 con 0x7f09c41095d0 2024-04-17T07:58:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.034+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f09c402d820 con 0x7f09c41095d0 2024-04-17T07:58:44.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.035+0000 7f09c2000640 1 -- 172.21.15.46:0/3024845104 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f09b809d1f0 con 0x7f09c41095d0 2024-04-17T07:58:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09ac0a3c30 msgr2=0x7f09ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09ac0a3c30 0x7f09ac0a60f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09b40058c0 tx=0x7f09b4005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c41095d0 msgr2=0x7f09c419ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c41095d0 0x7f09c419ebf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009fd0 tx=0x7f09b800ef90 comp rx=0 tx=0).stop 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 shutdown_connections 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09ac0a3c30 0x7f09ac0a60f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c4109f90 0x7f09c41a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09c41095d0 0x7f09c419ebf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 --2- 172.21.15.46:0/3024845104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c41083e0 0x7f09c419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 >> 172.21.15.46:0/3024845104 conn(0x7f09c40773a0 msgr2=0x7f09c410c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 shutdown_connections 2024-04-17T07:58:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.038+0000 7f09ca800640 1 -- 172.21.15.46:0/3024845104 wait complete. 2024-04-17T07:58:44.058 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-17T07:58:44.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:58:44.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 -- 172.21.15.46:0/3105841218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b601096a0 msgr2=0x7f1b60109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 --2- 172.21.15.46:0/3105841218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b601096a0 0x7f1b60109b20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009900 tx=0x7f1b4802bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:44.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 -- 172.21.15.46:0/3105841218 shutdown_connections 2024-04-17T07:58:44.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 --2- 172.21.15.46:0/3105841218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6010a060 0x7f1b601108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 --2- 172.21.15.46:0/3105841218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b601096a0 0x7f1b60109b20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 --2- 172.21.15.46:0/3105841218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b60108500 0x7f1b60108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 -- 172.21.15.46:0/3105841218 >> 172.21.15.46:0/3105841218 conn(0x7f1b60077450 msgr2=0x7f1b60079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 -- 172.21.15.46:0/3105841218 shutdown_connections 2024-04-17T07:58:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.281+0000 7f1b64a00640 1 -- 172.21.15.46:0/3105841218 wait complete. 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 Processor -- start 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 -- start start 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b60108500 0x7f1b6019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b601096a0 0x7f1b6019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6010a060 0x7f1b601a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60114140 con 0x7f1b6010a060 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60113fc0 con 0x7f1b60108500 2024-04-17T07:58:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.282+0000 7f1b64a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601142c0 con 0x7f1b601096a0 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b5f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6010a060 0x7f1b601a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b57e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b601096a0 0x7f1b6019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b5f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6010a060 0x7f1b601a5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60630/0 (socket says 172.21.15.46:60630) 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b57e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b601096a0 0x7f1b6019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:38452/0 (socket says 172.21.15.46:38452) 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b5f200640 1 -- 172.21.15.46:0/362198588 learned_addr learned my addr 172.21.15.46:0/362198588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b5e800640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b60108500 0x7f1b6019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b57e00640 1 -- 172.21.15.46:0/362198588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b60108500 msgr2=0x7f1b6019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b57e00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b60108500 0x7f1b6019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b57e00640 1 -- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6010a060 msgr2=0x7f1b601a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b57e00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6010a060 0x7f1b601a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b57e00640 1 -- 172.21.15.46:0/362198588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b601a63b0 con 0x7f1b601096a0 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b5f200640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6010a060 0x7f1b601a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.283+0000 7f1b5e800640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b60108500 0x7f1b6019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.284+0000 7f1b57e00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b601096a0 0x7f1b6019ecd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009230 tx=0x7f1b4802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.284+0000 7f1b57400640 1 -- 172.21.15.46:0/362198588 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48041070 con 0x7f1b601096a0 2024-04-17T07:58:44.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.284+0000 7f1b57400640 1 -- 172.21.15.46:0/362198588 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b48004450 con 0x7f1b601096a0 2024-04-17T07:58:44.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.284+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b601a6640 con 0x7f1b601096a0 2024-04-17T07:58:44.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.284+0000 7f1b57400640 1 -- 172.21.15.46:0/362198588 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48003e50 con 0x7f1b601096a0 2024-04-17T07:58:44.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.284+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601a6b80 con 0x7f1b601096a0 2024-04-17T07:58:44.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.285+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6010f7d0 con 0x7f1b601096a0 2024-04-17T07:58:44.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.287+0000 7f1b57400640 1 -- 172.21.15.46:0/362198588 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1b48044070 con 0x7f1b601096a0 2024-04-17T07:58:44.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.289+0000 7f1b57400640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1b440a3dc0 0x7f1b440a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.289+0000 7f1b57400640 1 -- 172.21.15.46:0/362198588 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f1b480f2320 con 0x7f1b601096a0 2024-04-17T07:58:44.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.289+0000 7f1b5e800640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1b440a3dc0 0x7f1b440a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.290+0000 7f1b5e800640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1b440a3dc0 0x7f1b440a6280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b60109500 tx=0x7f1b50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:44.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.291+0000 7f1b57400640 1 -- 172.21.15.46:0/362198588 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1b480be2c0 con 0x7f1b601096a0 2024-04-17T07:58:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.540+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b6002d5d0 con 0x7f1b601096a0 2024-04-17T07:58:44.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.542+0000 7f1b57400640 1 -- 172.21.15.46:0/362198588 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7f1b480c25c0 con 0x7f1b601096a0 2024-04-17T07:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:44.541 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:58:39.019766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T07:58:44.542 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.544+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1b440a3dc0 msgr2=0x7f1b440a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.544+0000 7f1b64a00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1b440a3dc0 0x7f1b440a6280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1b60109500 tx=0x7f1b50005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b601096a0 msgr2=0x7f1b6019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b601096a0 0x7f1b6019ecd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009230 tx=0x7f1b4802c660 comp rx=0 tx=0).stop 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 shutdown_connections 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f1b440a3dc0 0x7f1b440a6280 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b6010a060 0x7f1b601a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b601096a0 0x7f1b6019ecd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 --2- 172.21.15.46:0/362198588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b60108500 0x7f1b6019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 >> 172.21.15.46:0/362198588 conn(0x7f1b60077450 msgr2=0x7f1b6010c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:44.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 shutdown_connections 2024-04-17T07:58:44.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.545+0000 7f1b64a00640 1 -- 172.21.15.46:0/362198588 wait complete. 2024-04-17T07:58:44.566 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-17T07:58:44.566 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-17T07:58:44.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:58:44.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:44 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2291311875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:44.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:44 smithi046 ceph-mon[32038]: pgmap v424: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-17T07:58:44.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:44 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3024845104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:44.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:44 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2291311875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:44.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:44 smithi053 ceph-mon[36437]: pgmap v424: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-17T07:58:44.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:44 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3024845104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.793+0000 7f632fe00640 1 -- 172.21.15.46:0/1954604300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 msgr2=0x7f6330104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.793+0000 7f632fe00640 1 --2- 172.21.15.46:0/1954604300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f6330104d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6320009900 tx=0x7f632002bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.794+0000 7f632fe00640 1 -- 172.21.15.46:0/1954604300 shutdown_connections 2024-04-17T07:58:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.794+0000 7f632fe00640 1 --2- 172.21.15.46:0/1954604300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6330107b80 0x7f6330109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.794+0000 7f632fe00640 1 --2- 172.21.15.46:0/1954604300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6330105250 0x7f6330107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.794+0000 7f632fe00640 1 --2- 172.21.15.46:0/1954604300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f6330104d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.794+0000 7f632fe00640 1 -- 172.21.15.46:0/1954604300 >> 172.21.15.46:0/1954604300 conn(0x7f63300fc330 msgr2=0x7f63300fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.794+0000 7f632fe00640 1 -- 172.21.15.46:0/1954604300 shutdown_connections 2024-04-17T07:58:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.794+0000 7f632fe00640 1 -- 172.21.15.46:0/1954604300 wait complete. 2024-04-17T07:58:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.795+0000 7f632fe00640 1 Processor -- start 2024-04-17T07:58:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.795+0000 7f632fe00640 1 -- start start 2024-04-17T07:58:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f633019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6330105250 0x7f633019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6330107b80 0x7f63301a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f633010db90 con 0x7f6330107b80 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f633010da10 con 0x7f6330105250 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f633010dd10 con 0x7f6330102920 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f633019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f633019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:38460/0 (socket says 172.21.15.46:38460) 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632ea00640 1 -- 172.21.15.46:0/615947446 learned_addr learned my addr 172.21.15.46:0/615947446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632f400640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6330107b80 0x7f63301a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632e000640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6330105250 0x7f633019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632ea00640 1 -- 172.21.15.46:0/615947446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6330105250 msgr2=0x7f633019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632ea00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6330105250 0x7f633019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632ea00640 1 -- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6330107b80 msgr2=0x7f63301a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.796+0000 7f632ea00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6330107b80 0x7f63301a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f632ea00640 1 -- 172.21.15.46:0/615947446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63301a2020 con 0x7f6330102920 2024-04-17T07:58:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f632f400640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6330107b80 0x7f63301a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f632ea00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f633019a400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f632002c8f0 tx=0x7f632002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f6327400640 1 -- 172.21.15.46:0/615947446 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320041070 con 0x7f6330102920 2024-04-17T07:58:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f6327400640 1 -- 172.21.15.46:0/615947446 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6320004450 con 0x7f6330102920 2024-04-17T07:58:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63301a22b0 con 0x7f6330102920 2024-04-17T07:58:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f6327400640 1 -- 172.21.15.46:0/615947446 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320034440 con 0x7f6330102920 2024-04-17T07:58:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.797+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63301a27f0 con 0x7f6330102920 2024-04-17T07:58:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.798+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63301096d0 con 0x7f6330102920 2024-04-17T07:58:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.800+0000 7f6327400640 1 -- 172.21.15.46:0/615947446 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6320044070 con 0x7f6330102920 2024-04-17T07:58:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.802+0000 7f6327400640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63100a3c20 0x7f63100a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.802+0000 7f6327400640 1 -- 172.21.15.46:0/615947446 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f63200f2650 con 0x7f6330102920 2024-04-17T07:58:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.802+0000 7f632e000640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63100a3c20 0x7f63100a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.803+0000 7f632e000640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63100a3c20 0x7f63100a60e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f633019b910 tx=0x7f6318009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:44.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:44 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2291311875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:44.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:44 smithi062 ceph-mon[36507]: pgmap v424: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-17T07:58:44.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:44 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3024845104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:44.804+0000 7f6327400640 1 -- 172.21.15.46:0/615947446 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f63200be5f0 con 0x7f6330102920 2024-04-17T07:58:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.005+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63300623a0 con 0x7f63100a3c20 2024-04-17T07:58:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.019+0000 7f6327400640 1 -- 172.21.15.46:0/615947446 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340496 (secure 0 0 0) 0x7f63300623a0 con 0x7f63100a3c20 2024-04-17T07:58:45.023 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:45.025 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:58:45.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.024+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63100a3c20 msgr2=0x7f63100a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.024+0000 7f632fe00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63100a3c20 0x7f63100a60e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f633019b910 tx=0x7f6318009500 comp rx=0 tx=0).stop 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.024+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 msgr2=0x7f633019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.024+0000 7f632fe00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f633019a400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f632002c8f0 tx=0x7f632002c660 comp rx=0 tx=0).stop 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 shutdown_connections 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f63100a3c20 0x7f63100a60e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6330107b80 0x7f63301a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6330105250 0x7f633019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 --2- 172.21.15.46:0/615947446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6330102920 0x7f633019a400 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 >> 172.21.15.46:0/615947446 conn(0x7f63300fc330 msgr2=0x7f63301020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 shutdown_connections 2024-04-17T07:58:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.025+0000 7f632fe00640 1 -- 172.21.15.46:0/615947446 wait complete. 2024-04-17T07:58:45.045 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-04-17T07:58:43.592451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663833,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1161,"num_read_kb":707,"num_write":362,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181973,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380340,"kb_used_data":14488,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561484,"statfs":{"total":1151940427776,"available":1151550959616,"internally_reserved":0,"allocated":14835712,"data_stored":8197124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5683,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005406"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987034+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:57:22.987034+0000","last_peered":"2024-04-17T07:57:22.987034+0000","last_clean":"2024-04-17T07:57:22.987034+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:57:22.987034+0000","last_undegraded":"2024-04-17T07:57:22.987034+0000","last_fullsized":"2024-04-17T07:57:22.987034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301182+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:57:23.301182+0000","last_peered":"2024-04-17T07:57:23.301182+0000","last_clean":"2024-04-17T07:57:23.301182+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:57:23.301182+0000","last_undegraded":"2024-04-17T07:57:23.301182+0000","last_fullsized":"2024-04-17T07:57:23.301182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300823+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:57:23.300823+0000","last_peered":"2024-04-17T07:57:23.300823+0000","last_clean":"2024-04-17T07:57:23.300823+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:57:23.300823+0000","last_undegraded":"2024-04-17T07:57:23.300823+0000","last_fullsized":"2024-04-17T07:57:23.300823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987715+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:57:22.987715+0000","last_peered":"2024-04-17T07:57:22.987715+0000","last_clean":"2024-04-17T07:57:22.987715+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:57:22.987715+0000","last_undegraded":"2024-04-17T07:57:22.987715+0000","last_fullsized":"2024-04-17T07:57:22.987715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989625+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:57:22.989625+0000","last_peered":"2024-04-17T07:57:22.989625+0000","last_clean":"2024-04-17T07:57:22.989625+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:57:22.989625+0000","last_undegraded":"2024-04-17T07:57:22.989625+0000","last_fullsized":"2024-04-17T07:57:22.989625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990264+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:57:22.990264+0000","last_peered":"2024-04-17T07:57:22.990264+0000","last_clean":"2024-04-17T07:57:22.990264+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:57:22.990264+0000","last_undegraded":"2024-04-17T07:57:22.990264+0000","last_fullsized":"2024-04-17T07:57:22.990264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989501+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:57:22.989501+0000","last_peered":"2024-04-17T07:57:22.989501+0000","last_clean":"2024-04-17T07:57:22.989501+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:57:22.989501+0000","last_undegraded":"2024-04-17T07:57:22.989501+0000","last_fullsized":"2024-04-17T07:57:22.989501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981451+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:57:22.981451+0000","last_peered":"2024-04-17T07:57:22.981451+0000","last_clean":"2024-04-17T07:57:22.981451+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:57:22.981451+0000","last_undegraded":"2024-04-17T07:57:22.981451+0000","last_fullsized":"2024-04-17T07:57:22.981451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298435+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:57:23.298435+0000","last_peered":"2024-04-17T07:57:23.298435+0000","last_clean":"2024-04-17T07:57:23.298435+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:57:23.298435+0000","last_undegraded":"2024-04-17T07:57:23.298435+0000","last_fullsized":"2024-04-17T07:57:23.298435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991068+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:57:22.991068+0000","last_peered":"2024-04-17T07:57:22.991068+0000","last_clean":"2024-04-17T07:57:22.991068+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:57:22.991068+0000","last_undegraded":"2024-04-17T07:57:22.991068+0000","last_fullsized":"2024-04-17T07:57:22.991068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990259+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:57:22.990259+0000","last_peered":"2024-04-17T07:57:22.990259+0000","last_clean":"2024-04-17T07:57:22.990259+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:57:22.990259+0000","last_undegraded":"2024-04-17T07:57:22.990259+0000","last_fullsized":"2024-04-17T07:57:22.990259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987830+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:57:22.987830+0000","last_peered":"2024-04-17T07:57:22.987830+0000","last_clean":"2024-04-17T07:57:22.987830+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:57:22.987830+0000","last_undegraded":"2024-04-17T07:57:22.987830+0000","last_fullsized":"2024-04-17T07:57:22.987830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984491+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:57:22.984491+0000","last_peered":"2024-04-17T07:57:22.984491+0000","last_clean":"2024-04-17T07:57:22.984491+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:57:22.984491+0000","last_undegraded":"2024-04-17T07:57:22.984491+0000","last_fullsized":"2024-04-17T07:57:22.984491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300180+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:57:23.300180+0000","last_peered":"2024-04-17T07:57:23.300180+0000","last_clean":"2024-04-17T07:57:23.300180+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:57:23.300180+0000","last_undegraded":"2024-04-17T07:57:23.300180+0000","last_fullsized":"2024-04-17T07:57:23.300180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300300+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:57:23.300300+0000","last_peered":"2024-04-17T07:57:23.300300+0000","last_clean":"2024-04-17T07:57:23.300300+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:57:23.300300+0000","last_undegraded":"2024-04-17T07:57:23.300300+0000","last_fullsized":"2024-04-17T07:57:23.300300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297670+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:57:23.297670+0000","last_peered":"2024-04-17T07:57:23.297670+0000","last_clean":"2024-04-17T07:57:23.297670+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:57:23.297670+0000","last_undegraded":"2024-04-17T07:57:23.297670+0000","last_fullsized":"2024-04-17T07:57:23.297670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986912+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:57:22.986912+0000","last_peered":"2024-04-17T07:57:22.986912+0000","last_clean":"2024-04-17T07:57:22.986912+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:57:22.986912+0000","last_undegraded":"2024-04-17T07:57:22.986912+0000","last_fullsized":"2024-04-17T07:57:22.986912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299088+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:57:23.299088+0000","last_peered":"2024-04-17T07:57:23.299088+0000","last_clean":"2024-04-17T07:57:23.299088+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:57:23.299088+0000","last_undegraded":"2024-04-17T07:57:23.299088+0000","last_fullsized":"2024-04-17T07:57:23.299088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990554+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:57:22.990554+0000","last_peered":"2024-04-17T07:57:22.990554+0000","last_clean":"2024-04-17T07:57:22.990554+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:57:22.990554+0000","last_undegraded":"2024-04-17T07:57:22.990554+0000","last_fullsized":"2024-04-17T07:57:22.990554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990152+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:57:22.990152+0000","last_peered":"2024-04-17T07:57:22.990152+0000","last_clean":"2024-04-17T07:57:22.990152+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:57:22.990152+0000","last_undegraded":"2024-04-17T07:57:22.990152+0000","last_fullsized":"2024-04-17T07:57:22.990152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299637+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:57:23.299637+0000","last_peered":"2024-04-17T07:57:23.299637+0000","last_clean":"2024-04-17T07:57:23.299637+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:57:23.299637+0000","last_undegraded":"2024-04-17T07:57:23.299637+0000","last_fullsized":"2024-04-17T07:57:23.299637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298966+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:57:23.298966+0000","last_peered":"2024-04-17T07:57:23.298966+0000","last_clean":"2024-04-17T07:57:23.298966+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:57:23.298966+0000","last_undegraded":"2024-04-17T07:57:23.298966+0000","last_fullsized":"2024-04-17T07:57:23.298966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987127+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:57:22.987127+0000","last_peered":"2024-04-17T07:57:22.987127+0000","last_clean":"2024-04-17T07:57:22.987127+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:57:22.987127+0000","last_undegraded":"2024-04-17T07:57:22.987127+0000","last_fullsized":"2024-04-17T07:57:22.987127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984617+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:57:22.984617+0000","last_peered":"2024-04-17T07:57:22.984617+0000","last_clean":"2024-04-17T07:57:22.984617+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:57:22.984617+0000","last_undegraded":"2024-04-17T07:57:22.984617+0000","last_fullsized":"2024-04-17T07:57:22.984617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301222+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:57:23.301222+0000","last_peered":"2024-04-17T07:57:23.301222+0000","last_clean":"2024-04-17T07:57:23.301222+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:57:23.301222+0000","last_undegraded":"2024-04-17T07:57:23.301222+0000","last_fullsized":"2024-04-17T07:57:23.301222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987203+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:57:22.987203+0000","last_peered":"2024-04-17T07:57:22.987203+0000","last_clean":"2024-04-17T07:57:22.987203+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:57:22.987203+0000","last_undegraded":"2024-04-17T07:57:22.987203+0000","last_fullsized":"2024-04-17T07:57:22.987203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.996476+0000","last_change":"2024-04-17T07:57:22.996476+0000","last_active":"2024-04-17T07:57:22.996476+0000","last_peered":"2024-04-17T07:57:22.996476+0000","last_clean":"2024-04-17T07:57:22.996476+0000","last_became_active":"2024-04-17T07:57:22.991654+0000","last_became_peered":"2024-04-17T07:57:22.991654+0000","last_unstale":"2024-04-17T07:57:22.996476+0000","last_undegraded":"2024-04-17T07:57:22.996476+0000","last_fullsized":"2024-04-17T07:57:22.996476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:33.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298058+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:57:23.298058+0000","last_peered":"2024-04-17T07:57:23.298058+0000","last_clean":"2024-04-17T07:57:23.298058+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:57:23.298058+0000","last_undegraded":"2024-04-17T07:57:23.298058+0000","last_fullsized":"2024-04-17T07:57:23.298058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987151+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:57:22.987151+0000","last_peered":"2024-04-17T07:57:22.987151+0000","last_clean":"2024-04-17T07:57:22.987151+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:57:22.987151+0000","last_undegraded":"2024-04-17T07:57:22.987151+0000","last_fullsized":"2024-04-17T07:57:22.987151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990261+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:57:22.990261+0000","last_peered":"2024-04-17T07:57:22.990261+0000","last_clean":"2024-04-17T07:57:22.990261+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:57:22.990261+0000","last_undegraded":"2024-04-17T07:57:22.990261+0000","last_fullsized":"2024-04-17T07:57:22.990261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982016+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:57:22.982016+0000","last_peered":"2024-04-17T07:57:22.982016+0000","last_clean":"2024-04-17T07:57:22.982016+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:57:22.982016+0000","last_undegraded":"2024-04-17T07:57:22.982016+0000","last_fullsized":"2024-04-17T07:57:22.982016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990411+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:57:22.990411+0000","last_peered":"2024-04-17T07:57:22.990411+0000","last_clean":"2024-04-17T07:57:22.990411+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:57:22.990411+0000","last_undegraded":"2024-04-17T07:57:22.990411+0000","last_fullsized":"2024-04-17T07:57:22.990411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":210,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:39.804590+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:58:39.804590+0000","last_peered":"2024-04-17T07:58:39.804590+0000","last_clean":"2024-04-17T07:58:39.804590+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:58:39.804590+0000","last_undegraded":"2024-04-17T07:58:39.804590+0000","last_fullsized":"2024-04-17T07:58:39.804590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990055+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:57:22.990055+0000","last_peered":"2024-04-17T07:57:22.990055+0000","last_clean":"2024-04-17T07:57:22.990055+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:57:22.990055+0000","last_undegraded":"2024-04-17T07:57:22.990055+0000","last_fullsized":"2024-04-17T07:57:22.990055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989753+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:57:22.989753+0000","last_peered":"2024-04-17T07:57:22.989753+0000","last_clean":"2024-04-17T07:57:22.989753+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:57:22.989753+0000","last_undegraded":"2024-04-17T07:57:22.989753+0000","last_fullsized":"2024-04-17T07:57:22.989753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990141+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:57:22.990141+0000","last_peered":"2024-04-17T07:57:22.990141+0000","last_clean":"2024-04-17T07:57:22.990141+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:57:22.990141+0000","last_undegraded":"2024-04-17T07:57:22.990141+0000","last_fullsized":"2024-04-17T07:57:22.990141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986902+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:57:22.986902+0000","last_peered":"2024-04-17T07:57:22.986902+0000","last_clean":"2024-04-17T07:57:22.986902+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:57:22.986902+0000","last_undegraded":"2024-04-17T07:57:22.986902+0000","last_fullsized":"2024-04-17T07:57:22.986902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982901+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:57:22.982901+0000","last_peered":"2024-04-17T07:57:22.982901+0000","last_clean":"2024-04-17T07:57:22.982901+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:57:22.982901+0000","last_undegraded":"2024-04-17T07:57:22.982901+0000","last_fullsized":"2024-04-17T07:57:22.982901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981110+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:57:22.981110+0000","last_peered":"2024-04-17T07:57:22.981110+0000","last_clean":"2024-04-17T07:57:22.981110+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:57:22.981110+0000","last_undegraded":"2024-04-17T07:57:22.981110+0000","last_fullsized":"2024-04-17T07:57:22.981110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301002+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:57:23.301002+0000","last_peered":"2024-04-17T07:57:23.301002+0000","last_clean":"2024-04-17T07:57:23.301002+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:57:23.301002+0000","last_undegraded":"2024-04-17T07:57:23.301002+0000","last_fullsized":"2024-04-17T07:57:23.301002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981473+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:57:22.981473+0000","last_peered":"2024-04-17T07:57:22.981473+0000","last_clean":"2024-04-17T07:57:22.981473+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:57:22.981473+0000","last_undegraded":"2024-04-17T07:57:22.981473+0000","last_fullsized":"2024-04-17T07:57:22.981473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299094+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:57:23.299094+0000","last_peered":"2024-04-17T07:57:23.299094+0000","last_clean":"2024-04-17T07:57:23.299094+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:57:23.299094+0000","last_undegraded":"2024-04-17T07:57:23.299094+0000","last_fullsized":"2024-04-17T07:57:23.299094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981348+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:57:22.981348+0000","last_peered":"2024-04-17T07:57:22.981348+0000","last_clean":"2024-04-17T07:57:22.981348+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:57:22.981348+0000","last_undegraded":"2024-04-17T07:57:22.981348+0000","last_fullsized":"2024-04-17T07:57:22.981348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":204,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:40.805698+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:58:40.805698+0000","last_peered":"2024-04-17T07:58:40.805698+0000","last_clean":"2024-04-17T07:58:40.805698+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:58:40.805698+0000","last_undegraded":"2024-04-17T07:58:40.805698+0000","last_fullsized":"2024-04-17T07:58:40.805698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298436+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:57:23.298436+0000","last_peered":"2024-04-17T07:57:23.298436+0000","last_clean":"2024-04-17T07:57:23.298436+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:57:23.298436+0000","last_undegraded":"2024-04-17T07:57:23.298436+0000","last_fullsized":"2024-04-17T07:57:23.298436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299555+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:57:23.299555+0000","last_peered":"2024-04-17T07:57:23.299555+0000","last_clean":"2024-04-17T07:57:23.299555+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:57:23.299555+0000","last_undegraded":"2024-04-17T07:57:23.299555+0000","last_fullsized":"2024-04-17T07:57:23.299555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990114+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:57:22.990114+0000","last_peered":"2024-04-17T07:57:22.990114+0000","last_clean":"2024-04-17T07:57:22.990114+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:57:22.990114+0000","last_undegraded":"2024-04-17T07:57:22.990114+0000","last_fullsized":"2024-04-17T07:57:22.990114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301169+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:57:23.301169+0000","last_peered":"2024-04-17T07:57:23.301169+0000","last_clean":"2024-04-17T07:57:23.301169+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:57:23.301169+0000","last_undegraded":"2024-04-17T07:57:23.301169+0000","last_fullsized":"2024-04-17T07:57:23.301169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987333+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:57:22.987333+0000","last_peered":"2024-04-17T07:57:22.987333+0000","last_clean":"2024-04-17T07:57:22.987333+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:57:22.987333+0000","last_undegraded":"2024-04-17T07:57:22.987333+0000","last_fullsized":"2024-04-17T07:57:22.987333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981989+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:57:22.981989+0000","last_peered":"2024-04-17T07:57:22.981989+0000","last_clean":"2024-04-17T07:57:22.981989+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:57:22.981989+0000","last_undegraded":"2024-04-17T07:57:22.981989+0000","last_fullsized":"2024-04-17T07:57:22.981989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298180+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:57:23.298180+0000","last_peered":"2024-04-17T07:57:23.298180+0000","last_clean":"2024-04-17T07:57:23.298180+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:57:23.298180+0000","last_undegraded":"2024-04-17T07:57:23.298180+0000","last_fullsized":"2024-04-17T07:57:23.298180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298705+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:57:23.298705+0000","last_peered":"2024-04-17T07:57:23.298705+0000","last_clean":"2024-04-17T07:57:23.298705+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:57:23.298705+0000","last_undegraded":"2024-04-17T07:57:23.298705+0000","last_fullsized":"2024-04-17T07:57:23.298705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":417,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:41.808235+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:58:41.808235+0000","last_peered":"2024-04-17T07:58:41.808235+0000","last_clean":"2024-04-17T07:58:41.808235+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:58:41.808235+0000","last_undegraded":"2024-04-17T07:58:41.808235+0000","last_fullsized":"2024-04-17T07:58:41.808235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68557,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":347,"num_read_kb":242,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991109+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:57:22.991109+0000","last_peered":"2024-04-17T07:57:22.991109+0000","last_clean":"2024-04-17T07:57:22.991109+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:57:22.991109+0000","last_undegraded":"2024-04-17T07:57:22.991109+0000","last_fullsized":"2024-04-17T07:57:22.991109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982223+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:57:22.982223+0000","last_peered":"2024-04-17T07:57:22.982223+0000","last_clean":"2024-04-17T07:57:22.982223+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:57:22.982223+0000","last_undegraded":"2024-04-17T07:57:22.982223+0000","last_fullsized":"2024-04-17T07:57:22.982223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989456+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:57:22.989456+0000","last_peered":"2024-04-17T07:57:22.989456+0000","last_clean":"2024-04-17T07:57:22.989456+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:57:22.989456+0000","last_undegraded":"2024-04-17T07:57:22.989456+0000","last_fullsized":"2024-04-17T07:57:22.989456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983461+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:57:22.983461+0000","last_peered":"2024-04-17T07:57:22.983461+0000","last_clean":"2024-04-17T07:57:22.983461+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:57:22.983461+0000","last_undegraded":"2024-04-17T07:57:22.983461+0000","last_fullsized":"2024-04-17T07:57:22.983461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986452+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:57:22.986452+0000","last_peered":"2024-04-17T07:57:22.986452+0000","last_clean":"2024-04-17T07:57:22.986452+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:57:22.986452+0000","last_undegraded":"2024-04-17T07:57:22.986452+0000","last_fullsized":"2024-04-17T07:57:22.986452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981873+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:57:22.981873+0000","last_peered":"2024-04-17T07:57:22.981873+0000","last_clean":"2024-04-17T07:57:22.981873+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:57:22.981873+0000","last_undegraded":"2024-04-17T07:57:22.981873+0000","last_fullsized":"2024-04-17T07:57:22.981873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989629+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:57:22.989629+0000","last_peered":"2024-04-17T07:57:22.989629+0000","last_clean":"2024-04-17T07:57:22.989629+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:57:22.989629+0000","last_undegraded":"2024-04-17T07:57:22.989629+0000","last_fullsized":"2024-04-17T07:57:22.989629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299422+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:57:23.299422+0000","last_peered":"2024-04-17T07:57:23.299422+0000","last_clean":"2024-04-17T07:57:23.299422+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:57:23.299422+0000","last_undegraded":"2024-04-17T07:57:23.299422+0000","last_fullsized":"2024-04-17T07:57:23.299422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":219,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:39.166397+0000","last_change":"2024-04-17T07:57:22.995880+0000","last_active":"2024-04-17T07:58:39.166397+0000","last_peered":"2024-04-17T07:58:39.166397+0000","last_clean":"2024-04-17T07:58:39.166397+0000","last_became_active":"2024-04-17T07:57:22.991967+0000","last_became_peered":"2024-04-17T07:57:22.991967+0000","last_unstale":"2024-04-17T07:58:39.166397+0000","last_undegraded":"2024-04-17T07:58:39.166397+0000","last_fullsized":"2024-04-17T07:58:39.166397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:27.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989564+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:57:22.989564+0000","last_peered":"2024-04-17T07:57:22.989564+0000","last_clean":"2024-04-17T07:57:22.989564+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:57:22.989564+0000","last_undegraded":"2024-04-17T07:57:22.989564+0000","last_fullsized":"2024-04-17T07:57:22.989564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984648+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:57:22.984648+0000","last_peered":"2024-04-17T07:57:22.984648+0000","last_clean":"2024-04-17T07:57:22.984648+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:57:22.984648+0000","last_undegraded":"2024-04-17T07:57:22.984648+0000","last_fullsized":"2024-04-17T07:57:22.984648+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989830+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:57:22.989830+0000","last_peered":"2024-04-17T07:57:22.989830+0000","last_clean":"2024-04-17T07:57:22.989830+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:57:22.989830+0000","last_undegraded":"2024-04-17T07:57:22.989830+0000","last_fullsized":"2024-04-17T07:57:22.989830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981593+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:57:22.981593+0000","last_peered":"2024-04-17T07:57:22.981593+0000","last_clean":"2024-04-17T07:57:22.981593+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:57:22.981593+0000","last_undegraded":"2024-04-17T07:57:22.981593+0000","last_fullsized":"2024-04-17T07:57:22.981593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981392+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:57:22.981392+0000","last_peered":"2024-04-17T07:57:22.981392+0000","last_clean":"2024-04-17T07:57:22.981392+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:57:22.981392+0000","last_undegraded":"2024-04-17T07:57:22.981392+0000","last_fullsized":"2024-04-17T07:57:22.981392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991262+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:57:22.991262+0000","last_peered":"2024-04-17T07:57:22.991262+0000","last_clean":"2024-04-17T07:57:22.991262+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:57:22.991262+0000","last_undegraded":"2024-04-17T07:57:22.991262+0000","last_fullsized":"2024-04-17T07:57:22.991262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986868+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:57:22.986868+0000","last_peered":"2024-04-17T07:57:22.986868+0000","last_clean":"2024-04-17T07:57:22.986868+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:57:22.986868+0000","last_undegraded":"2024-04-17T07:57:22.986868+0000","last_fullsized":"2024-04-17T07:57:22.986868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989693+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:57:22.989693+0000","last_peered":"2024-04-17T07:57:22.989693+0000","last_clean":"2024-04-17T07:57:22.989693+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:57:22.989693+0000","last_undegraded":"2024-04-17T07:57:22.989693+0000","last_fullsized":"2024-04-17T07:57:22.989693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299647+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:57:23.299647+0000","last_peered":"2024-04-17T07:57:23.299647+0000","last_clean":"2024-04-17T07:57:23.299647+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:57:23.299647+0000","last_undegraded":"2024-04-17T07:57:23.299647+0000","last_fullsized":"2024-04-17T07:57:23.299647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982610+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:57:22.982610+0000","last_peered":"2024-04-17T07:57:22.982610+0000","last_clean":"2024-04-17T07:57:22.982610+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:57:22.982610+0000","last_undegraded":"2024-04-17T07:57:22.982610+0000","last_fullsized":"2024-04-17T07:57:22.982610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989416+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:57:22.989416+0000","last_peered":"2024-04-17T07:57:22.989416+0000","last_clean":"2024-04-17T07:57:22.989416+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:57:22.989416+0000","last_undegraded":"2024-04-17T07:57:22.989416+0000","last_fullsized":"2024-04-17T07:57:22.989416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990564+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:57:22.990564+0000","last_peered":"2024-04-17T07:57:22.990564+0000","last_clean":"2024-04-17T07:57:22.990564+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:57:22.990564+0000","last_undegraded":"2024-04-17T07:57:22.990564+0000","last_fullsized":"2024-04-17T07:57:22.990564+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.985794+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:57:22.985794+0000","last_peered":"2024-04-17T07:57:22.985794+0000","last_clean":"2024-04-17T07:57:22.985794+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:57:22.985794+0000","last_undegraded":"2024-04-17T07:57:22.985794+0000","last_fullsized":"2024-04-17T07:57:22.985794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":204,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:40.167483+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:58:40.167483+0000","last_peered":"2024-04-17T07:58:40.167483+0000","last_clean":"2024-04-17T07:58:40.167483+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:58:40.167483+0000","last_undegraded":"2024-04-17T07:58:40.167483+0000","last_fullsized":"2024-04-17T07:58:40.167483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989218+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:57:22.989218+0000","last_peered":"2024-04-17T07:57:22.989218+0000","last_clean":"2024-04-17T07:57:22.989218+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:57:22.989218+0000","last_undegraded":"2024-04-17T07:57:22.989218+0000","last_fullsized":"2024-04-17T07:57:22.989218+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982675+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:57:22.982675+0000","last_peered":"2024-04-17T07:57:22.982675+0000","last_clean":"2024-04-17T07:57:22.982675+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:57:22.982675+0000","last_undegraded":"2024-04-17T07:57:22.982675+0000","last_fullsized":"2024-04-17T07:57:22.982675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987305+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:57:22.987305+0000","last_peered":"2024-04-17T07:57:22.987305+0000","last_clean":"2024-04-17T07:57:22.987305+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:57:22.987305+0000","last_undegraded":"2024-04-17T07:57:22.987305+0000","last_fullsized":"2024-04-17T07:57:22.987305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989248+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:57:22.989248+0000","last_peered":"2024-04-17T07:57:22.989248+0000","last_clean":"2024-04-17T07:57:22.989248+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:57:22.989248+0000","last_undegraded":"2024-04-17T07:57:22.989248+0000","last_fullsized":"2024-04-17T07:57:22.989248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982074+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:57:22.982074+0000","last_peered":"2024-04-17T07:57:22.982074+0000","last_clean":"2024-04-17T07:57:22.982074+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:57:22.982074+0000","last_undegraded":"2024-04-17T07:57:22.982074+0000","last_fullsized":"2024-04-17T07:57:22.982074+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982417+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:57:22.982417+0000","last_peered":"2024-04-17T07:57:22.982417+0000","last_clean":"2024-04-17T07:57:22.982417+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:57:22.982417+0000","last_undegraded":"2024-04-17T07:57:22.982417+0000","last_fullsized":"2024-04-17T07:57:22.982417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989408+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:57:22.989408+0000","last_peered":"2024-04-17T07:57:22.989408+0000","last_clean":"2024-04-17T07:57:22.989408+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:57:22.989408+0000","last_undegraded":"2024-04-17T07:57:22.989408+0000","last_fullsized":"2024-04-17T07:57:22.989408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986358+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:57:22.986358+0000","last_peered":"2024-04-17T07:57:22.986358+0000","last_clean":"2024-04-17T07:57:22.986358+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:57:22.986358+0000","last_undegraded":"2024-04-17T07:57:22.986358+0000","last_fullsized":"2024-04-17T07:57:22.986358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983466+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:57:22.983466+0000","last_peered":"2024-04-17T07:57:22.983466+0000","last_clean":"2024-04-17T07:57:22.983466+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:57:22.983466+0000","last_undegraded":"2024-04-17T07:57:22.983466+0000","last_fullsized":"2024-04-17T07:57:22.983466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981116+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:57:22.981116+0000","last_peered":"2024-04-17T07:57:22.981116+0000","last_clean":"2024-04-17T07:57:22.981116+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:57:22.981116+0000","last_undegraded":"2024-04-17T07:57:22.981116+0000","last_fullsized":"2024-04-17T07:57:22.981116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'12","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:35.129665+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:57:35.129665+0000","last_peered":"2024-04-17T07:57:35.129665+0000","last_clean":"2024-04-17T07:57:35.129665+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:57:35.129665+0000","last_undegraded":"2024-04-17T07:57:35.129665+0000","last_fullsized":"2024-04-17T07:57:35.129665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984249+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:57:22.984249+0000","last_peered":"2024-04-17T07:57:22.984249+0000","last_clean":"2024-04-17T07:57:22.984249+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:57:22.984249+0000","last_undegraded":"2024-04-17T07:57:22.984249+0000","last_fullsized":"2024-04-17T07:57:22.984249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989826+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:57:22.989826+0000","last_peered":"2024-04-17T07:57:22.989826+0000","last_clean":"2024-04-17T07:57:22.989826+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:57:22.989826+0000","last_undegraded":"2024-04-17T07:57:22.989826+0000","last_fullsized":"2024-04-17T07:57:22.989826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987958+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:57:22.987958+0000","last_peered":"2024-04-17T07:57:22.987958+0000","last_clean":"2024-04-17T07:57:22.987958+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:57:22.987958+0000","last_undegraded":"2024-04-17T07:57:22.987958+0000","last_fullsized":"2024-04-17T07:57:22.987958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301351+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:57:23.301351+0000","last_peered":"2024-04-17T07:57:23.301351+0000","last_clean":"2024-04-17T07:57:23.301351+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:57:23.301351+0000","last_undegraded":"2024-04-17T07:57:23.301351+0000","last_fullsized":"2024-04-17T07:57:23.301351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987643+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:57:22.987643+0000","last_peered":"2024-04-17T07:57:22.987643+0000","last_clean":"2024-04-17T07:57:22.987643+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:57:22.987643+0000","last_undegraded":"2024-04-17T07:57:22.987643+0000","last_fullsized":"2024-04-17T07:57:22.987643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297743+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:57:23.297743+0000","last_peered":"2024-04-17T07:57:23.297743+0000","last_clean":"2024-04-17T07:57:23.297743+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:57:23.297743+0000","last_undegraded":"2024-04-17T07:57:23.297743+0000","last_fullsized":"2024-04-17T07:57:23.297743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989106+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:57:22.989106+0000","last_peered":"2024-04-17T07:57:22.989106+0000","last_clean":"2024-04-17T07:57:22.989106+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:57:22.989106+0000","last_undegraded":"2024-04-17T07:57:22.989106+0000","last_fullsized":"2024-04-17T07:57:22.989106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299584+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:57:23.299584+0000","last_peered":"2024-04-17T07:57:23.299584+0000","last_clean":"2024-04-17T07:57:23.299584+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:57:23.299584+0000","last_undegraded":"2024-04-17T07:57:23.299584+0000","last_fullsized":"2024-04-17T07:57:23.299584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981676+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:57:22.981676+0000","last_peered":"2024-04-17T07:57:22.981676+0000","last_clean":"2024-04-17T07:57:22.981676+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:57:22.981676+0000","last_undegraded":"2024-04-17T07:57:22.981676+0000","last_fullsized":"2024-04-17T07:57:22.981676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986346+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:57:22.986346+0000","last_peered":"2024-04-17T07:57:22.986346+0000","last_clean":"2024-04-17T07:57:22.986346+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:57:22.986346+0000","last_undegraded":"2024-04-17T07:57:22.986346+0000","last_fullsized":"2024-04-17T07:57:22.986346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987410+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:57:22.987410+0000","last_peered":"2024-04-17T07:57:22.987410+0000","last_clean":"2024-04-17T07:57:22.987410+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:57:22.987410+0000","last_undegraded":"2024-04-17T07:57:22.987410+0000","last_fullsized":"2024-04-17T07:57:22.987410+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990346+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:57:22.990346+0000","last_peered":"2024-04-17T07:57:22.990346+0000","last_clean":"2024-04-17T07:57:22.990346+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:57:22.990346+0000","last_undegraded":"2024-04-17T07:57:22.990346+0000","last_fullsized":"2024-04-17T07:57:22.990346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983638+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:57:22.983638+0000","last_peered":"2024-04-17T07:57:22.983638+0000","last_clean":"2024-04-17T07:57:22.983638+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:57:22.983638+0000","last_undegraded":"2024-04-17T07:57:22.983638+0000","last_fullsized":"2024-04-17T07:57:22.983638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987732+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:57:22.987732+0000","last_peered":"2024-04-17T07:57:22.987732+0000","last_clean":"2024-04-17T07:57:22.987732+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:57:22.987732+0000","last_undegraded":"2024-04-17T07:57:22.987732+0000","last_fullsized":"2024-04-17T07:57:22.987732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989966+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:57:22.989966+0000","last_peered":"2024-04-17T07:57:22.989966+0000","last_clean":"2024-04-17T07:57:22.989966+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:57:22.989966+0000","last_undegraded":"2024-04-17T07:57:22.989966+0000","last_fullsized":"2024-04-17T07:57:22.989966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989548+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:57:22.989548+0000","last_peered":"2024-04-17T07:57:22.989548+0000","last_clean":"2024-04-17T07:57:22.989548+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:57:22.989548+0000","last_undegraded":"2024-04-17T07:57:22.989548+0000","last_fullsized":"2024-04-17T07:57:22.989548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297780+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:57:23.297780+0000","last_peered":"2024-04-17T07:57:23.297780+0000","last_clean":"2024-04-17T07:57:23.297780+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:57:23.297780+0000","last_undegraded":"2024-04-17T07:57:23.297780+0000","last_fullsized":"2024-04-17T07:57:23.297780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990064+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:57:22.990064+0000","last_peered":"2024-04-17T07:57:22.990064+0000","last_clean":"2024-04-17T07:57:22.990064+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:57:22.990064+0000","last_undegraded":"2024-04-17T07:57:22.990064+0000","last_fullsized":"2024-04-17T07:57:22.990064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990382+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:57:22.990382+0000","last_peered":"2024-04-17T07:57:22.990382+0000","last_clean":"2024-04-17T07:57:22.990382+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:57:22.990382+0000","last_undegraded":"2024-04-17T07:57:22.990382+0000","last_fullsized":"2024-04-17T07:57:22.990382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299886+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:57:23.299886+0000","last_peered":"2024-04-17T07:57:23.299886+0000","last_clean":"2024-04-17T07:57:23.299886+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:57:23.299886+0000","last_undegraded":"2024-04-17T07:57:23.299886+0000","last_fullsized":"2024-04-17T07:57:23.299886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982144+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:57:22.982144+0000","last_peered":"2024-04-17T07:57:22.982144+0000","last_clean":"2024-04-17T07:57:22.982144+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:57:22.982144+0000","last_undegraded":"2024-04-17T07:57:22.982144+0000","last_fullsized":"2024-04-17T07:57:22.982144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989834+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:57:22.989834+0000","last_peered":"2024-04-17T07:57:22.989834+0000","last_clean":"2024-04-17T07:57:22.989834+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:57:22.989834+0000","last_undegraded":"2024-04-17T07:57:22.989834+0000","last_fullsized":"2024-04-17T07:57:22.989834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981278+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:57:22.981278+0000","last_peered":"2024-04-17T07:57:22.981278+0000","last_clean":"2024-04-17T07:57:22.981278+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:57:22.981278+0000","last_undegraded":"2024-04-17T07:57:22.981278+0000","last_fullsized":"2024-04-17T07:57:22.981278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984913+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:57:22.984913+0000","last_peered":"2024-04-17T07:57:22.984913+0000","last_clean":"2024-04-17T07:57:22.984913+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:57:22.984913+0000","last_undegraded":"2024-04-17T07:57:22.984913+0000","last_fullsized":"2024-04-17T07:57:22.984913+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989950+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:57:22.989950+0000","last_peered":"2024-04-17T07:57:22.989950+0000","last_clean":"2024-04-17T07:57:22.989950+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:57:22.989950+0000","last_undegraded":"2024-04-17T07:57:22.989950+0000","last_fullsized":"2024-04-17T07:57:22.989950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986499+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:57:22.986499+0000","last_peered":"2024-04-17T07:57:22.986499+0000","last_clean":"2024-04-17T07:57:22.986499+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:57:22.986499+0000","last_undegraded":"2024-04-17T07:57:22.986499+0000","last_fullsized":"2024-04-17T07:57:22.986499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990103+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:57:22.990103+0000","last_peered":"2024-04-17T07:57:22.990103+0000","last_clean":"2024-04-17T07:57:22.990103+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:57:22.990103+0000","last_undegraded":"2024-04-17T07:57:22.990103+0000","last_fullsized":"2024-04-17T07:57:22.990103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990216+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:57:22.990216+0000","last_peered":"2024-04-17T07:57:22.990216+0000","last_clean":"2024-04-17T07:57:22.990216+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:57:22.990216+0000","last_undegraded":"2024-04-17T07:57:22.990216+0000","last_fullsized":"2024-04-17T07:57:22.990216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300391+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:57:23.300391+0000","last_peered":"2024-04-17T07:57:23.300391+0000","last_clean":"2024-04-17T07:57:23.300391+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:57:23.300391+0000","last_undegraded":"2024-04-17T07:57:23.300391+0000","last_fullsized":"2024-04-17T07:57:23.300391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983162+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:57:22.983162+0000","last_peered":"2024-04-17T07:57:22.983162+0000","last_clean":"2024-04-17T07:57:22.983162+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:57:22.983162+0000","last_undegraded":"2024-04-17T07:57:22.983162+0000","last_fullsized":"2024-04-17T07:57:22.983162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986726+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:57:22.986726+0000","last_peered":"2024-04-17T07:57:22.986726+0000","last_clean":"2024-04-17T07:57:22.986726+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:57:22.986726+0000","last_undegraded":"2024-04-17T07:57:22.986726+0000","last_fullsized":"2024-04-17T07:57:22.986726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990569+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:57:22.990569+0000","last_peered":"2024-04-17T07:57:22.990569+0000","last_clean":"2024-04-17T07:57:22.990569+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:57:22.990569+0000","last_undegraded":"2024-04-17T07:57:22.990569+0000","last_fullsized":"2024-04-17T07:57:22.990569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984737+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:57:22.984737+0000","last_peered":"2024-04-17T07:57:22.984737+0000","last_clean":"2024-04-17T07:57:22.984737+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:57:22.984737+0000","last_undegraded":"2024-04-17T07:57:22.984737+0000","last_fullsized":"2024-04-17T07:57:22.984737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981744+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:57:22.981744+0000","last_peered":"2024-04-17T07:57:22.981744+0000","last_clean":"2024-04-17T07:57:22.981744+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:57:22.981744+0000","last_undegraded":"2024-04-17T07:57:22.981744+0000","last_fullsized":"2024-04-17T07:57:22.981744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981978+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:57:22.981978+0000","last_peered":"2024-04-17T07:57:22.981978+0000","last_clean":"2024-04-17T07:57:22.981978+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:57:22.981978+0000","last_undegraded":"2024-04-17T07:57:22.981978+0000","last_fullsized":"2024-04-17T07:57:22.981978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298052+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:57:23.298052+0000","last_peered":"2024-04-17T07:57:23.298052+0000","last_clean":"2024-04-17T07:57:23.298052+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:57:23.298052+0000","last_undegraded":"2024-04-17T07:57:23.298052+0000","last_fullsized":"2024-04-17T07:57:23.298052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984677+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:57:22.984677+0000","last_peered":"2024-04-17T07:57:22.984677+0000","last_clean":"2024-04-17T07:57:22.984677+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:57:22.984677+0000","last_undegraded":"2024-04-17T07:57:22.984677+0000","last_fullsized":"2024-04-17T07:57:22.984677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986573+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:57:22.986573+0000","last_peered":"2024-04-17T07:57:22.986573+0000","last_clean":"2024-04-17T07:57:22.986573+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:57:22.986573+0000","last_undegraded":"2024-04-17T07:57:22.986573+0000","last_fullsized":"2024-04-17T07:57:22.986573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73231,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1035,"num_read_kb":598,"num_write":179,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":219693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383735,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1175946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808896,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":523921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070534,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":523841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332172,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1115207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495698,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":578095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":585350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1107287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:58:45.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:58:45.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.318+0000 7f109f200640 1 -- 172.21.15.46:0/2587566690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a0105c70 msgr2=0x7f10a010c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.318+0000 7f109f200640 1 --2- 172.21.15.46:0/2587566690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a0105c70 0x7f10a010c500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1098009ca0 tx=0x7f109802c100 comp rx=0 tx=0).stop 2024-04-17T07:58:45.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.319+0000 7f109f200640 1 -- 172.21.15.46:0/2587566690 shutdown_connections 2024-04-17T07:58:45.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.319+0000 7f109f200640 1 --2- 172.21.15.46:0/2587566690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a0105c70 0x7f10a010c500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.319+0000 7f109f200640 1 --2- 172.21.15.46:0/2587566690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01052b0 0x7f10a0105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.319+0000 7f109f200640 1 --2- 172.21.15.46:0/2587566690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a01040c0 0x7f10a01044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.319+0000 7f109f200640 1 -- 172.21.15.46:0/2587566690 >> 172.21.15.46:0/2587566690 conn(0x7f10a00ff630 msgr2=0x7f10a0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:45.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.319+0000 7f109f200640 1 -- 172.21.15.46:0/2587566690 shutdown_connections 2024-04-17T07:58:45.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.319+0000 7f109f200640 1 -- 172.21.15.46:0/2587566690 wait complete. 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 Processor -- start 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 -- start start 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01040c0 0x7f10a019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a01052b0 0x7f10a019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a0105c70 0x7f10a01a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10a010fd50 con 0x7f10a0105c70 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10a010fbd0 con 0x7f10a01040c0 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.320+0000 7f109f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10a010fed0 con 0x7f10a01052b0 2024-04-17T07:58:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a0105c70 0x7f10a01a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01040c0 0x7f10a019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f1095400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a01052b0 0x7f10a019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a0105c70 0x7f10a01a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60666/0 (socket says 172.21.15.46:60666) 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109de00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01040c0 0x7f10a019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50140/0 (socket says 172.21.15.46:50140) 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 -- 172.21.15.46:0/2548906587 learned_addr learned my addr 172.21.15.46:0/2548906587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 -- 172.21.15.46:0/2548906587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a01052b0 msgr2=0x7f10a019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a01052b0 0x7f10a019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 -- 172.21.15.46:0/2548906587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01040c0 msgr2=0x7f10a019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01040c0 0x7f10a019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 -- 172.21.15.46:0/2548906587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a01a1f90 con 0x7f10a0105c70 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f1095400640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a01052b0 0x7f10a019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109de00640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01040c0 0x7f10a019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.321+0000 7f109e800640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a0105c70 0x7f10a01a1920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f109802c640 tx=0x7f1098035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.322+0000 7f1097400640 1 -- 172.21.15.46:0/2548906587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098039040 con 0x7f10a0105c70 2024-04-17T07:58:45.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.322+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a01a2220 con 0x7f10a0105c70 2024-04-17T07:58:45.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.322+0000 7f1097400640 1 -- 172.21.15.46:0/2548906587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1098044070 con 0x7f10a0105c70 2024-04-17T07:58:45.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.322+0000 7f1097400640 1 -- 172.21.15.46:0/2548906587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10980043c0 con 0x7f10a0105c70 2024-04-17T07:58:45.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.322+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a01a2760 con 0x7f10a0105c70 2024-04-17T07:58:45.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.323+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a010b380 con 0x7f10a0105c70 2024-04-17T07:58:45.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.325+0000 7f1097400640 1 -- 172.21.15.46:0/2548906587 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1098004560 con 0x7f10a0105c70 2024-04-17T07:58:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.327+0000 7f1097400640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f10800a3ba0 0x7f10800a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.327+0000 7f1097400640 1 -- 172.21.15.46:0/2548906587 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f10980f3130 con 0x7f10a0105c70 2024-04-17T07:58:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.327+0000 7f109de00640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f10800a3ba0 0x7f10800a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.328+0000 7f109de00640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f10800a3ba0 0x7f10800a6060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f10a0105110 tx=0x7f1090005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.330+0000 7f1097400640 1 -- 172.21.15.46:0/2548906587 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f109804e020 con 0x7f10a0105c70 2024-04-17T07:58:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.537+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10a00623a0 con 0x7f10800a3ba0 2024-04-17T07:58:45.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.551+0000 7f1097400640 1 -- 172.21.15.46:0/2548906587 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340496 (secure 0 0 0) 0x7f10a00623a0 con 0x7f10800a3ba0 2024-04-17T07:58:45.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:45.557 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:58:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f10800a3ba0 msgr2=0x7f10800a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f10800a3ba0 0x7f10800a6060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f10a0105110 tx=0x7f1090005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a0105c70 msgr2=0x7f10a01a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a0105c70 0x7f10a01a1920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f109802c640 tx=0x7f1098035d10 comp rx=0 tx=0).stop 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 shutdown_connections 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f10800a3ba0 0x7f10800a6060 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a0105c70 0x7f10a01a1920 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10a01052b0 0x7f10a019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 --2- 172.21.15.46:0/2548906587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10a01040c0 0x7f10a019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.558+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 >> 172.21.15.46:0/2548906587 conn(0x7f10a00ff630 msgr2=0x7f10a0108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.559+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 shutdown_connections 2024-04-17T07:58:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.559+0000 7f109f200640 1 -- 172.21.15.46:0/2548906587 wait complete. 2024-04-17T07:58:45.578 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-04-17T07:58:43.592451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663833,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1161,"num_read_kb":707,"num_write":362,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181973,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380340,"kb_used_data":14488,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561484,"statfs":{"total":1151940427776,"available":1151550959616,"internally_reserved":0,"allocated":14835712,"data_stored":8197124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5683,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005406"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987034+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:57:22.987034+0000","last_peered":"2024-04-17T07:57:22.987034+0000","last_clean":"2024-04-17T07:57:22.987034+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:57:22.987034+0000","last_undegraded":"2024-04-17T07:57:22.987034+0000","last_fullsized":"2024-04-17T07:57:22.987034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301182+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:57:23.301182+0000","last_peered":"2024-04-17T07:57:23.301182+0000","last_clean":"2024-04-17T07:57:23.301182+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:57:23.301182+0000","last_undegraded":"2024-04-17T07:57:23.301182+0000","last_fullsized":"2024-04-17T07:57:23.301182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300823+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:57:23.300823+0000","last_peered":"2024-04-17T07:57:23.300823+0000","last_clean":"2024-04-17T07:57:23.300823+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:57:23.300823+0000","last_undegraded":"2024-04-17T07:57:23.300823+0000","last_fullsized":"2024-04-17T07:57:23.300823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987715+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:57:22.987715+0000","last_peered":"2024-04-17T07:57:22.987715+0000","last_clean":"2024-04-17T07:57:22.987715+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:57:22.987715+0000","last_undegraded":"2024-04-17T07:57:22.987715+0000","last_fullsized":"2024-04-17T07:57:22.987715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989625+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:57:22.989625+0000","last_peered":"2024-04-17T07:57:22.989625+0000","last_clean":"2024-04-17T07:57:22.989625+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:57:22.989625+0000","last_undegraded":"2024-04-17T07:57:22.989625+0000","last_fullsized":"2024-04-17T07:57:22.989625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990264+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:57:22.990264+0000","last_peered":"2024-04-17T07:57:22.990264+0000","last_clean":"2024-04-17T07:57:22.990264+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:57:22.990264+0000","last_undegraded":"2024-04-17T07:57:22.990264+0000","last_fullsized":"2024-04-17T07:57:22.990264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989501+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:57:22.989501+0000","last_peered":"2024-04-17T07:57:22.989501+0000","last_clean":"2024-04-17T07:57:22.989501+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:57:22.989501+0000","last_undegraded":"2024-04-17T07:57:22.989501+0000","last_fullsized":"2024-04-17T07:57:22.989501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981451+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:57:22.981451+0000","last_peered":"2024-04-17T07:57:22.981451+0000","last_clean":"2024-04-17T07:57:22.981451+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:57:22.981451+0000","last_undegraded":"2024-04-17T07:57:22.981451+0000","last_fullsized":"2024-04-17T07:57:22.981451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298435+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:57:23.298435+0000","last_peered":"2024-04-17T07:57:23.298435+0000","last_clean":"2024-04-17T07:57:23.298435+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:57:23.298435+0000","last_undegraded":"2024-04-17T07:57:23.298435+0000","last_fullsized":"2024-04-17T07:57:23.298435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991068+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:57:22.991068+0000","last_peered":"2024-04-17T07:57:22.991068+0000","last_clean":"2024-04-17T07:57:22.991068+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:57:22.991068+0000","last_undegraded":"2024-04-17T07:57:22.991068+0000","last_fullsized":"2024-04-17T07:57:22.991068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990259+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:57:22.990259+0000","last_peered":"2024-04-17T07:57:22.990259+0000","last_clean":"2024-04-17T07:57:22.990259+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:57:22.990259+0000","last_undegraded":"2024-04-17T07:57:22.990259+0000","last_fullsized":"2024-04-17T07:57:22.990259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987830+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:57:22.987830+0000","last_peered":"2024-04-17T07:57:22.987830+0000","last_clean":"2024-04-17T07:57:22.987830+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:57:22.987830+0000","last_undegraded":"2024-04-17T07:57:22.987830+0000","last_fullsized":"2024-04-17T07:57:22.987830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984491+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:57:22.984491+0000","last_peered":"2024-04-17T07:57:22.984491+0000","last_clean":"2024-04-17T07:57:22.984491+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:57:22.984491+0000","last_undegraded":"2024-04-17T07:57:22.984491+0000","last_fullsized":"2024-04-17T07:57:22.984491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300180+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:57:23.300180+0000","last_peered":"2024-04-17T07:57:23.300180+0000","last_clean":"2024-04-17T07:57:23.300180+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:57:23.300180+0000","last_undegraded":"2024-04-17T07:57:23.300180+0000","last_fullsized":"2024-04-17T07:57:23.300180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300300+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:57:23.300300+0000","last_peered":"2024-04-17T07:57:23.300300+0000","last_clean":"2024-04-17T07:57:23.300300+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:57:23.300300+0000","last_undegraded":"2024-04-17T07:57:23.300300+0000","last_fullsized":"2024-04-17T07:57:23.300300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297670+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:57:23.297670+0000","last_peered":"2024-04-17T07:57:23.297670+0000","last_clean":"2024-04-17T07:57:23.297670+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:57:23.297670+0000","last_undegraded":"2024-04-17T07:57:23.297670+0000","last_fullsized":"2024-04-17T07:57:23.297670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986912+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:57:22.986912+0000","last_peered":"2024-04-17T07:57:22.986912+0000","last_clean":"2024-04-17T07:57:22.986912+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:57:22.986912+0000","last_undegraded":"2024-04-17T07:57:22.986912+0000","last_fullsized":"2024-04-17T07:57:22.986912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299088+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:57:23.299088+0000","last_peered":"2024-04-17T07:57:23.299088+0000","last_clean":"2024-04-17T07:57:23.299088+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:57:23.299088+0000","last_undegraded":"2024-04-17T07:57:23.299088+0000","last_fullsized":"2024-04-17T07:57:23.299088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990554+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:57:22.990554+0000","last_peered":"2024-04-17T07:57:22.990554+0000","last_clean":"2024-04-17T07:57:22.990554+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:57:22.990554+0000","last_undegraded":"2024-04-17T07:57:22.990554+0000","last_fullsized":"2024-04-17T07:57:22.990554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990152+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:57:22.990152+0000","last_peered":"2024-04-17T07:57:22.990152+0000","last_clean":"2024-04-17T07:57:22.990152+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:57:22.990152+0000","last_undegraded":"2024-04-17T07:57:22.990152+0000","last_fullsized":"2024-04-17T07:57:22.990152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299637+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:57:23.299637+0000","last_peered":"2024-04-17T07:57:23.299637+0000","last_clean":"2024-04-17T07:57:23.299637+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:57:23.299637+0000","last_undegraded":"2024-04-17T07:57:23.299637+0000","last_fullsized":"2024-04-17T07:57:23.299637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298966+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:57:23.298966+0000","last_peered":"2024-04-17T07:57:23.298966+0000","last_clean":"2024-04-17T07:57:23.298966+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:57:23.298966+0000","last_undegraded":"2024-04-17T07:57:23.298966+0000","last_fullsized":"2024-04-17T07:57:23.298966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987127+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:57:22.987127+0000","last_peered":"2024-04-17T07:57:22.987127+0000","last_clean":"2024-04-17T07:57:22.987127+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:57:22.987127+0000","last_undegraded":"2024-04-17T07:57:22.987127+0000","last_fullsized":"2024-04-17T07:57:22.987127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984617+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:57:22.984617+0000","last_peered":"2024-04-17T07:57:22.984617+0000","last_clean":"2024-04-17T07:57:22.984617+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:57:22.984617+0000","last_undegraded":"2024-04-17T07:57:22.984617+0000","last_fullsized":"2024-04-17T07:57:22.984617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301222+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:57:23.301222+0000","last_peered":"2024-04-17T07:57:23.301222+0000","last_clean":"2024-04-17T07:57:23.301222+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:57:23.301222+0000","last_undegraded":"2024-04-17T07:57:23.301222+0000","last_fullsized":"2024-04-17T07:57:23.301222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987203+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:57:22.987203+0000","last_peered":"2024-04-17T07:57:22.987203+0000","last_clean":"2024-04-17T07:57:22.987203+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:57:22.987203+0000","last_undegraded":"2024-04-17T07:57:22.987203+0000","last_fullsized":"2024-04-17T07:57:22.987203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.996476+0000","last_change":"2024-04-17T07:57:22.996476+0000","last_active":"2024-04-17T07:57:22.996476+0000","last_peered":"2024-04-17T07:57:22.996476+0000","last_clean":"2024-04-17T07:57:22.996476+0000","last_became_active":"2024-04-17T07:57:22.991654+0000","last_became_peered":"2024-04-17T07:57:22.991654+0000","last_unstale":"2024-04-17T07:57:22.996476+0000","last_undegraded":"2024-04-17T07:57:22.996476+0000","last_fullsized":"2024-04-17T07:57:22.996476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:33.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298058+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:57:23.298058+0000","last_peered":"2024-04-17T07:57:23.298058+0000","last_clean":"2024-04-17T07:57:23.298058+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:57:23.298058+0000","last_undegraded":"2024-04-17T07:57:23.298058+0000","last_fullsized":"2024-04-17T07:57:23.298058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987151+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:57:22.987151+0000","last_peered":"2024-04-17T07:57:22.987151+0000","last_clean":"2024-04-17T07:57:22.987151+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:57:22.987151+0000","last_undegraded":"2024-04-17T07:57:22.987151+0000","last_fullsized":"2024-04-17T07:57:22.987151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990261+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:57:22.990261+0000","last_peered":"2024-04-17T07:57:22.990261+0000","last_clean":"2024-04-17T07:57:22.990261+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:57:22.990261+0000","last_undegraded":"2024-04-17T07:57:22.990261+0000","last_fullsized":"2024-04-17T07:57:22.990261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982016+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:57:22.982016+0000","last_peered":"2024-04-17T07:57:22.982016+0000","last_clean":"2024-04-17T07:57:22.982016+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:57:22.982016+0000","last_undegraded":"2024-04-17T07:57:22.982016+0000","last_fullsized":"2024-04-17T07:57:22.982016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990411+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:57:22.990411+0000","last_peered":"2024-04-17T07:57:22.990411+0000","last_clean":"2024-04-17T07:57:22.990411+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:57:22.990411+0000","last_undegraded":"2024-04-17T07:57:22.990411+0000","last_fullsized":"2024-04-17T07:57:22.990411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":210,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:39.804590+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:58:39.804590+0000","last_peered":"2024-04-17T07:58:39.804590+0000","last_clean":"2024-04-17T07:58:39.804590+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:58:39.804590+0000","last_undegraded":"2024-04-17T07:58:39.804590+0000","last_fullsized":"2024-04-17T07:58:39.804590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990055+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:57:22.990055+0000","last_peered":"2024-04-17T07:57:22.990055+0000","last_clean":"2024-04-17T07:57:22.990055+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:57:22.990055+0000","last_undegraded":"2024-04-17T07:57:22.990055+0000","last_fullsized":"2024-04-17T07:57:22.990055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989753+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:57:22.989753+0000","last_peered":"2024-04-17T07:57:22.989753+0000","last_clean":"2024-04-17T07:57:22.989753+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:57:22.989753+0000","last_undegraded":"2024-04-17T07:57:22.989753+0000","last_fullsized":"2024-04-17T07:57:22.989753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990141+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:57:22.990141+0000","last_peered":"2024-04-17T07:57:22.990141+0000","last_clean":"2024-04-17T07:57:22.990141+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:57:22.990141+0000","last_undegraded":"2024-04-17T07:57:22.990141+0000","last_fullsized":"2024-04-17T07:57:22.990141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986902+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:57:22.986902+0000","last_peered":"2024-04-17T07:57:22.986902+0000","last_clean":"2024-04-17T07:57:22.986902+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:57:22.986902+0000","last_undegraded":"2024-04-17T07:57:22.986902+0000","last_fullsized":"2024-04-17T07:57:22.986902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982901+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:57:22.982901+0000","last_peered":"2024-04-17T07:57:22.982901+0000","last_clean":"2024-04-17T07:57:22.982901+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:57:22.982901+0000","last_undegraded":"2024-04-17T07:57:22.982901+0000","last_fullsized":"2024-04-17T07:57:22.982901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981110+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:57:22.981110+0000","last_peered":"2024-04-17T07:57:22.981110+0000","last_clean":"2024-04-17T07:57:22.981110+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:57:22.981110+0000","last_undegraded":"2024-04-17T07:57:22.981110+0000","last_fullsized":"2024-04-17T07:57:22.981110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301002+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:57:23.301002+0000","last_peered":"2024-04-17T07:57:23.301002+0000","last_clean":"2024-04-17T07:57:23.301002+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:57:23.301002+0000","last_undegraded":"2024-04-17T07:57:23.301002+0000","last_fullsized":"2024-04-17T07:57:23.301002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981473+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:57:22.981473+0000","last_peered":"2024-04-17T07:57:22.981473+0000","last_clean":"2024-04-17T07:57:22.981473+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:57:22.981473+0000","last_undegraded":"2024-04-17T07:57:22.981473+0000","last_fullsized":"2024-04-17T07:57:22.981473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299094+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:57:23.299094+0000","last_peered":"2024-04-17T07:57:23.299094+0000","last_clean":"2024-04-17T07:57:23.299094+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:57:23.299094+0000","last_undegraded":"2024-04-17T07:57:23.299094+0000","last_fullsized":"2024-04-17T07:57:23.299094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981348+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:57:22.981348+0000","last_peered":"2024-04-17T07:57:22.981348+0000","last_clean":"2024-04-17T07:57:22.981348+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:57:22.981348+0000","last_undegraded":"2024-04-17T07:57:22.981348+0000","last_fullsized":"2024-04-17T07:57:22.981348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":204,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:40.805698+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:58:40.805698+0000","last_peered":"2024-04-17T07:58:40.805698+0000","last_clean":"2024-04-17T07:58:40.805698+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:58:40.805698+0000","last_undegraded":"2024-04-17T07:58:40.805698+0000","last_fullsized":"2024-04-17T07:58:40.805698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298436+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:57:23.298436+0000","last_peered":"2024-04-17T07:57:23.298436+0000","last_clean":"2024-04-17T07:57:23.298436+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:57:23.298436+0000","last_undegraded":"2024-04-17T07:57:23.298436+0000","last_fullsized":"2024-04-17T07:57:23.298436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299555+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:57:23.299555+0000","last_peered":"2024-04-17T07:57:23.299555+0000","last_clean":"2024-04-17T07:57:23.299555+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:57:23.299555+0000","last_undegraded":"2024-04-17T07:57:23.299555+0000","last_fullsized":"2024-04-17T07:57:23.299555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990114+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:57:22.990114+0000","last_peered":"2024-04-17T07:57:22.990114+0000","last_clean":"2024-04-17T07:57:22.990114+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:57:22.990114+0000","last_undegraded":"2024-04-17T07:57:22.990114+0000","last_fullsized":"2024-04-17T07:57:22.990114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301169+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:57:23.301169+0000","last_peered":"2024-04-17T07:57:23.301169+0000","last_clean":"2024-04-17T07:57:23.301169+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:57:23.301169+0000","last_undegraded":"2024-04-17T07:57:23.301169+0000","last_fullsized":"2024-04-17T07:57:23.301169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987333+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:57:22.987333+0000","last_peered":"2024-04-17T07:57:22.987333+0000","last_clean":"2024-04-17T07:57:22.987333+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:57:22.987333+0000","last_undegraded":"2024-04-17T07:57:22.987333+0000","last_fullsized":"2024-04-17T07:57:22.987333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981989+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:57:22.981989+0000","last_peered":"2024-04-17T07:57:22.981989+0000","last_clean":"2024-04-17T07:57:22.981989+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:57:22.981989+0000","last_undegraded":"2024-04-17T07:57:22.981989+0000","last_fullsized":"2024-04-17T07:57:22.981989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298180+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:57:23.298180+0000","last_peered":"2024-04-17T07:57:23.298180+0000","last_clean":"2024-04-17T07:57:23.298180+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:57:23.298180+0000","last_undegraded":"2024-04-17T07:57:23.298180+0000","last_fullsized":"2024-04-17T07:57:23.298180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298705+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:57:23.298705+0000","last_peered":"2024-04-17T07:57:23.298705+0000","last_clean":"2024-04-17T07:57:23.298705+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:57:23.298705+0000","last_undegraded":"2024-04-17T07:57:23.298705+0000","last_fullsized":"2024-04-17T07:57:23.298705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":417,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:41.808235+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:58:41.808235+0000","last_peered":"2024-04-17T07:58:41.808235+0000","last_clean":"2024-04-17T07:58:41.808235+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:58:41.808235+0000","last_undegraded":"2024-04-17T07:58:41.808235+0000","last_fullsized":"2024-04-17T07:58:41.808235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68557,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":347,"num_read_kb":242,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991109+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:57:22.991109+0000","last_peered":"2024-04-17T07:57:22.991109+0000","last_clean":"2024-04-17T07:57:22.991109+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:57:22.991109+0000","last_undegraded":"2024-04-17T07:57:22.991109+0000","last_fullsized":"2024-04-17T07:57:22.991109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982223+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:57:22.982223+0000","last_peered":"2024-04-17T07:57:22.982223+0000","last_clean":"2024-04-17T07:57:22.982223+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:57:22.982223+0000","last_undegraded":"2024-04-17T07:57:22.982223+0000","last_fullsized":"2024-04-17T07:57:22.982223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989456+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:57:22.989456+0000","last_peered":"2024-04-17T07:57:22.989456+0000","last_clean":"2024-04-17T07:57:22.989456+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:57:22.989456+0000","last_undegraded":"2024-04-17T07:57:22.989456+0000","last_fullsized":"2024-04-17T07:57:22.989456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983461+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:57:22.983461+0000","last_peered":"2024-04-17T07:57:22.983461+0000","last_clean":"2024-04-17T07:57:22.983461+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:57:22.983461+0000","last_undegraded":"2024-04-17T07:57:22.983461+0000","last_fullsized":"2024-04-17T07:57:22.983461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986452+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:57:22.986452+0000","last_peered":"2024-04-17T07:57:22.986452+0000","last_clean":"2024-04-17T07:57:22.986452+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:57:22.986452+0000","last_undegraded":"2024-04-17T07:57:22.986452+0000","last_fullsized":"2024-04-17T07:57:22.986452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981873+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:57:22.981873+0000","last_peered":"2024-04-17T07:57:22.981873+0000","last_clean":"2024-04-17T07:57:22.981873+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:57:22.981873+0000","last_undegraded":"2024-04-17T07:57:22.981873+0000","last_fullsized":"2024-04-17T07:57:22.981873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989629+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:57:22.989629+0000","last_peered":"2024-04-17T07:57:22.989629+0000","last_clean":"2024-04-17T07:57:22.989629+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:57:22.989629+0000","last_undegraded":"2024-04-17T07:57:22.989629+0000","last_fullsized":"2024-04-17T07:57:22.989629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299422+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:57:23.299422+0000","last_peered":"2024-04-17T07:57:23.299422+0000","last_clean":"2024-04-17T07:57:23.299422+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:57:23.299422+0000","last_undegraded":"2024-04-17T07:57:23.299422+0000","last_fullsized":"2024-04-17T07:57:23.299422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":219,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:39.166397+0000","last_change":"2024-04-17T07:57:22.995880+0000","last_active":"2024-04-17T07:58:39.166397+0000","last_peered":"2024-04-17T07:58:39.166397+0000","last_clean":"2024-04-17T07:58:39.166397+0000","last_became_active":"2024-04-17T07:57:22.991967+0000","last_became_peered":"2024-04-17T07:57:22.991967+0000","last_unstale":"2024-04-17T07:58:39.166397+0000","last_undegraded":"2024-04-17T07:58:39.166397+0000","last_fullsized":"2024-04-17T07:58:39.166397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:27.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989564+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:57:22.989564+0000","last_peered":"2024-04-17T07:57:22.989564+0000","last_clean":"2024-04-17T07:57:22.989564+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:57:22.989564+0000","last_undegraded":"2024-04-17T07:57:22.989564+0000","last_fullsized":"2024-04-17T07:57:22.989564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984648+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:57:22.984648+0000","last_peered":"2024-04-17T07:57:22.984648+0000","last_clean":"2024-04-17T07:57:22.984648+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:57:22.984648+0000","last_undegraded":"2024-04-17T07:57:22.984648+0000","last_fullsized":"2024-04-17T07:57:22.984648+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989830+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:57:22.989830+0000","last_peered":"2024-04-17T07:57:22.989830+0000","last_clean":"2024-04-17T07:57:22.989830+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:57:22.989830+0000","last_undegraded":"2024-04-17T07:57:22.989830+0000","last_fullsized":"2024-04-17T07:57:22.989830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981593+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:57:22.981593+0000","last_peered":"2024-04-17T07:57:22.981593+0000","last_clean":"2024-04-17T07:57:22.981593+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:57:22.981593+0000","last_undegraded":"2024-04-17T07:57:22.981593+0000","last_fullsized":"2024-04-17T07:57:22.981593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981392+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:57:22.981392+0000","last_peered":"2024-04-17T07:57:22.981392+0000","last_clean":"2024-04-17T07:57:22.981392+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:57:22.981392+0000","last_undegraded":"2024-04-17T07:57:22.981392+0000","last_fullsized":"2024-04-17T07:57:22.981392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991262+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:57:22.991262+0000","last_peered":"2024-04-17T07:57:22.991262+0000","last_clean":"2024-04-17T07:57:22.991262+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:57:22.991262+0000","last_undegraded":"2024-04-17T07:57:22.991262+0000","last_fullsized":"2024-04-17T07:57:22.991262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986868+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:57:22.986868+0000","last_peered":"2024-04-17T07:57:22.986868+0000","last_clean":"2024-04-17T07:57:22.986868+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:57:22.986868+0000","last_undegraded":"2024-04-17T07:57:22.986868+0000","last_fullsized":"2024-04-17T07:57:22.986868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989693+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:57:22.989693+0000","last_peered":"2024-04-17T07:57:22.989693+0000","last_clean":"2024-04-17T07:57:22.989693+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:57:22.989693+0000","last_undegraded":"2024-04-17T07:57:22.989693+0000","last_fullsized":"2024-04-17T07:57:22.989693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299647+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:57:23.299647+0000","last_peered":"2024-04-17T07:57:23.299647+0000","last_clean":"2024-04-17T07:57:23.299647+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:57:23.299647+0000","last_undegraded":"2024-04-17T07:57:23.299647+0000","last_fullsized":"2024-04-17T07:57:23.299647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982610+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:57:22.982610+0000","last_peered":"2024-04-17T07:57:22.982610+0000","last_clean":"2024-04-17T07:57:22.982610+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:57:22.982610+0000","last_undegraded":"2024-04-17T07:57:22.982610+0000","last_fullsized":"2024-04-17T07:57:22.982610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989416+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:57:22.989416+0000","last_peered":"2024-04-17T07:57:22.989416+0000","last_clean":"2024-04-17T07:57:22.989416+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:57:22.989416+0000","last_undegraded":"2024-04-17T07:57:22.989416+0000","last_fullsized":"2024-04-17T07:57:22.989416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990564+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:57:22.990564+0000","last_peered":"2024-04-17T07:57:22.990564+0000","last_clean":"2024-04-17T07:57:22.990564+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:57:22.990564+0000","last_undegraded":"2024-04-17T07:57:22.990564+0000","last_fullsized":"2024-04-17T07:57:22.990564+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.985794+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:57:22.985794+0000","last_peered":"2024-04-17T07:57:22.985794+0000","last_clean":"2024-04-17T07:57:22.985794+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:57:22.985794+0000","last_undegraded":"2024-04-17T07:57:22.985794+0000","last_fullsized":"2024-04-17T07:57:22.985794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":204,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:40.167483+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:58:40.167483+0000","last_peered":"2024-04-17T07:58:40.167483+0000","last_clean":"2024-04-17T07:58:40.167483+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:58:40.167483+0000","last_undegraded":"2024-04-17T07:58:40.167483+0000","last_fullsized":"2024-04-17T07:58:40.167483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989218+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:57:22.989218+0000","last_peered":"2024-04-17T07:57:22.989218+0000","last_clean":"2024-04-17T07:57:22.989218+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:57:22.989218+0000","last_undegraded":"2024-04-17T07:57:22.989218+0000","last_fullsized":"2024-04-17T07:57:22.989218+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982675+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:57:22.982675+0000","last_peered":"2024-04-17T07:57:22.982675+0000","last_clean":"2024-04-17T07:57:22.982675+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:57:22.982675+0000","last_undegraded":"2024-04-17T07:57:22.982675+0000","last_fullsized":"2024-04-17T07:57:22.982675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987305+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:57:22.987305+0000","last_peered":"2024-04-17T07:57:22.987305+0000","last_clean":"2024-04-17T07:57:22.987305+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:57:22.987305+0000","last_undegraded":"2024-04-17T07:57:22.987305+0000","last_fullsized":"2024-04-17T07:57:22.987305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989248+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:57:22.989248+0000","last_peered":"2024-04-17T07:57:22.989248+0000","last_clean":"2024-04-17T07:57:22.989248+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:57:22.989248+0000","last_undegraded":"2024-04-17T07:57:22.989248+0000","last_fullsized":"2024-04-17T07:57:22.989248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982074+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:57:22.982074+0000","last_peered":"2024-04-17T07:57:22.982074+0000","last_clean":"2024-04-17T07:57:22.982074+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:57:22.982074+0000","last_undegraded":"2024-04-17T07:57:22.982074+0000","last_fullsized":"2024-04-17T07:57:22.982074+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982417+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:57:22.982417+0000","last_peered":"2024-04-17T07:57:22.982417+0000","last_clean":"2024-04-17T07:57:22.982417+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:57:22.982417+0000","last_undegraded":"2024-04-17T07:57:22.982417+0000","last_fullsized":"2024-04-17T07:57:22.982417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989408+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:57:22.989408+0000","last_peered":"2024-04-17T07:57:22.989408+0000","last_clean":"2024-04-17T07:57:22.989408+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:57:22.989408+0000","last_undegraded":"2024-04-17T07:57:22.989408+0000","last_fullsized":"2024-04-17T07:57:22.989408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986358+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:57:22.986358+0000","last_peered":"2024-04-17T07:57:22.986358+0000","last_clean":"2024-04-17T07:57:22.986358+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:57:22.986358+0000","last_undegraded":"2024-04-17T07:57:22.986358+0000","last_fullsized":"2024-04-17T07:57:22.986358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983466+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:57:22.983466+0000","last_peered":"2024-04-17T07:57:22.983466+0000","last_clean":"2024-04-17T07:57:22.983466+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:57:22.983466+0000","last_undegraded":"2024-04-17T07:57:22.983466+0000","last_fullsized":"2024-04-17T07:57:22.983466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981116+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:57:22.981116+0000","last_peered":"2024-04-17T07:57:22.981116+0000","last_clean":"2024-04-17T07:57:22.981116+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:57:22.981116+0000","last_undegraded":"2024-04-17T07:57:22.981116+0000","last_fullsized":"2024-04-17T07:57:22.981116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'12","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:35.129665+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:57:35.129665+0000","last_peered":"2024-04-17T07:57:35.129665+0000","last_clean":"2024-04-17T07:57:35.129665+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:57:35.129665+0000","last_undegraded":"2024-04-17T07:57:35.129665+0000","last_fullsized":"2024-04-17T07:57:35.129665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984249+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:57:22.984249+0000","last_peered":"2024-04-17T07:57:22.984249+0000","last_clean":"2024-04-17T07:57:22.984249+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:57:22.984249+0000","last_undegraded":"2024-04-17T07:57:22.984249+0000","last_fullsized":"2024-04-17T07:57:22.984249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989826+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:57:22.989826+0000","last_peered":"2024-04-17T07:57:22.989826+0000","last_clean":"2024-04-17T07:57:22.989826+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:57:22.989826+0000","last_undegraded":"2024-04-17T07:57:22.989826+0000","last_fullsized":"2024-04-17T07:57:22.989826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987958+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:57:22.987958+0000","last_peered":"2024-04-17T07:57:22.987958+0000","last_clean":"2024-04-17T07:57:22.987958+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:57:22.987958+0000","last_undegraded":"2024-04-17T07:57:22.987958+0000","last_fullsized":"2024-04-17T07:57:22.987958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301351+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:57:23.301351+0000","last_peered":"2024-04-17T07:57:23.301351+0000","last_clean":"2024-04-17T07:57:23.301351+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:57:23.301351+0000","last_undegraded":"2024-04-17T07:57:23.301351+0000","last_fullsized":"2024-04-17T07:57:23.301351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987643+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:57:22.987643+0000","last_peered":"2024-04-17T07:57:22.987643+0000","last_clean":"2024-04-17T07:57:22.987643+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:57:22.987643+0000","last_undegraded":"2024-04-17T07:57:22.987643+0000","last_fullsized":"2024-04-17T07:57:22.987643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297743+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:57:23.297743+0000","last_peered":"2024-04-17T07:57:23.297743+0000","last_clean":"2024-04-17T07:57:23.297743+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:57:23.297743+0000","last_undegraded":"2024-04-17T07:57:23.297743+0000","last_fullsized":"2024-04-17T07:57:23.297743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989106+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:57:22.989106+0000","last_peered":"2024-04-17T07:57:22.989106+0000","last_clean":"2024-04-17T07:57:22.989106+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:57:22.989106+0000","last_undegraded":"2024-04-17T07:57:22.989106+0000","last_fullsized":"2024-04-17T07:57:22.989106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299584+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:57:23.299584+0000","last_peered":"2024-04-17T07:57:23.299584+0000","last_clean":"2024-04-17T07:57:23.299584+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:57:23.299584+0000","last_undegraded":"2024-04-17T07:57:23.299584+0000","last_fullsized":"2024-04-17T07:57:23.299584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981676+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:57:22.981676+0000","last_peered":"2024-04-17T07:57:22.981676+0000","last_clean":"2024-04-17T07:57:22.981676+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:57:22.981676+0000","last_undegraded":"2024-04-17T07:57:22.981676+0000","last_fullsized":"2024-04-17T07:57:22.981676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986346+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:57:22.986346+0000","last_peered":"2024-04-17T07:57:22.986346+0000","last_clean":"2024-04-17T07:57:22.986346+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:57:22.986346+0000","last_undegraded":"2024-04-17T07:57:22.986346+0000","last_fullsized":"2024-04-17T07:57:22.986346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987410+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:57:22.987410+0000","last_peered":"2024-04-17T07:57:22.987410+0000","last_clean":"2024-04-17T07:57:22.987410+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:57:22.987410+0000","last_undegraded":"2024-04-17T07:57:22.987410+0000","last_fullsized":"2024-04-17T07:57:22.987410+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990346+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:57:22.990346+0000","last_peered":"2024-04-17T07:57:22.990346+0000","last_clean":"2024-04-17T07:57:22.990346+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:57:22.990346+0000","last_undegraded":"2024-04-17T07:57:22.990346+0000","last_fullsized":"2024-04-17T07:57:22.990346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983638+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:57:22.983638+0000","last_peered":"2024-04-17T07:57:22.983638+0000","last_clean":"2024-04-17T07:57:22.983638+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:57:22.983638+0000","last_undegraded":"2024-04-17T07:57:22.983638+0000","last_fullsized":"2024-04-17T07:57:22.983638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987732+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:57:22.987732+0000","last_peered":"2024-04-17T07:57:22.987732+0000","last_clean":"2024-04-17T07:57:22.987732+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:57:22.987732+0000","last_undegraded":"2024-04-17T07:57:22.987732+0000","last_fullsized":"2024-04-17T07:57:22.987732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989966+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:57:22.989966+0000","last_peered":"2024-04-17T07:57:22.989966+0000","last_clean":"2024-04-17T07:57:22.989966+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:57:22.989966+0000","last_undegraded":"2024-04-17T07:57:22.989966+0000","last_fullsized":"2024-04-17T07:57:22.989966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989548+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:57:22.989548+0000","last_peered":"2024-04-17T07:57:22.989548+0000","last_clean":"2024-04-17T07:57:22.989548+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:57:22.989548+0000","last_undegraded":"2024-04-17T07:57:22.989548+0000","last_fullsized":"2024-04-17T07:57:22.989548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297780+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:57:23.297780+0000","last_peered":"2024-04-17T07:57:23.297780+0000","last_clean":"2024-04-17T07:57:23.297780+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:57:23.297780+0000","last_undegraded":"2024-04-17T07:57:23.297780+0000","last_fullsized":"2024-04-17T07:57:23.297780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990064+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:57:22.990064+0000","last_peered":"2024-04-17T07:57:22.990064+0000","last_clean":"2024-04-17T07:57:22.990064+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:57:22.990064+0000","last_undegraded":"2024-04-17T07:57:22.990064+0000","last_fullsized":"2024-04-17T07:57:22.990064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990382+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:57:22.990382+0000","last_peered":"2024-04-17T07:57:22.990382+0000","last_clean":"2024-04-17T07:57:22.990382+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:57:22.990382+0000","last_undegraded":"2024-04-17T07:57:22.990382+0000","last_fullsized":"2024-04-17T07:57:22.990382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299886+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:57:23.299886+0000","last_peered":"2024-04-17T07:57:23.299886+0000","last_clean":"2024-04-17T07:57:23.299886+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:57:23.299886+0000","last_undegraded":"2024-04-17T07:57:23.299886+0000","last_fullsized":"2024-04-17T07:57:23.299886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982144+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:57:22.982144+0000","last_peered":"2024-04-17T07:57:22.982144+0000","last_clean":"2024-04-17T07:57:22.982144+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:57:22.982144+0000","last_undegraded":"2024-04-17T07:57:22.982144+0000","last_fullsized":"2024-04-17T07:57:22.982144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989834+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:57:22.989834+0000","last_peered":"2024-04-17T07:57:22.989834+0000","last_clean":"2024-04-17T07:57:22.989834+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:57:22.989834+0000","last_undegraded":"2024-04-17T07:57:22.989834+0000","last_fullsized":"2024-04-17T07:57:22.989834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981278+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:57:22.981278+0000","last_peered":"2024-04-17T07:57:22.981278+0000","last_clean":"2024-04-17T07:57:22.981278+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:57:22.981278+0000","last_undegraded":"2024-04-17T07:57:22.981278+0000","last_fullsized":"2024-04-17T07:57:22.981278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984913+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:57:22.984913+0000","last_peered":"2024-04-17T07:57:22.984913+0000","last_clean":"2024-04-17T07:57:22.984913+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:57:22.984913+0000","last_undegraded":"2024-04-17T07:57:22.984913+0000","last_fullsized":"2024-04-17T07:57:22.984913+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989950+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:57:22.989950+0000","last_peered":"2024-04-17T07:57:22.989950+0000","last_clean":"2024-04-17T07:57:22.989950+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:57:22.989950+0000","last_undegraded":"2024-04-17T07:57:22.989950+0000","last_fullsized":"2024-04-17T07:57:22.989950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986499+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:57:22.986499+0000","last_peered":"2024-04-17T07:57:22.986499+0000","last_clean":"2024-04-17T07:57:22.986499+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:57:22.986499+0000","last_undegraded":"2024-04-17T07:57:22.986499+0000","last_fullsized":"2024-04-17T07:57:22.986499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990103+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:57:22.990103+0000","last_peered":"2024-04-17T07:57:22.990103+0000","last_clean":"2024-04-17T07:57:22.990103+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:57:22.990103+0000","last_undegraded":"2024-04-17T07:57:22.990103+0000","last_fullsized":"2024-04-17T07:57:22.990103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990216+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:57:22.990216+0000","last_peered":"2024-04-17T07:57:22.990216+0000","last_clean":"2024-04-17T07:57:22.990216+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:57:22.990216+0000","last_undegraded":"2024-04-17T07:57:22.990216+0000","last_fullsized":"2024-04-17T07:57:22.990216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300391+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:57:23.300391+0000","last_peered":"2024-04-17T07:57:23.300391+0000","last_clean":"2024-04-17T07:57:23.300391+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:57:23.300391+0000","last_undegraded":"2024-04-17T07:57:23.300391+0000","last_fullsized":"2024-04-17T07:57:23.300391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983162+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:57:22.983162+0000","last_peered":"2024-04-17T07:57:22.983162+0000","last_clean":"2024-04-17T07:57:22.983162+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:57:22.983162+0000","last_undegraded":"2024-04-17T07:57:22.983162+0000","last_fullsized":"2024-04-17T07:57:22.983162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986726+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:57:22.986726+0000","last_peered":"2024-04-17T07:57:22.986726+0000","last_clean":"2024-04-17T07:57:22.986726+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:57:22.986726+0000","last_undegraded":"2024-04-17T07:57:22.986726+0000","last_fullsized":"2024-04-17T07:57:22.986726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990569+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:57:22.990569+0000","last_peered":"2024-04-17T07:57:22.990569+0000","last_clean":"2024-04-17T07:57:22.990569+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:57:22.990569+0000","last_undegraded":"2024-04-17T07:57:22.990569+0000","last_fullsized":"2024-04-17T07:57:22.990569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984737+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:57:22.984737+0000","last_peered":"2024-04-17T07:57:22.984737+0000","last_clean":"2024-04-17T07:57:22.984737+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:57:22.984737+0000","last_undegraded":"2024-04-17T07:57:22.984737+0000","last_fullsized":"2024-04-17T07:57:22.984737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981744+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:57:22.981744+0000","last_peered":"2024-04-17T07:57:22.981744+0000","last_clean":"2024-04-17T07:57:22.981744+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:57:22.981744+0000","last_undegraded":"2024-04-17T07:57:22.981744+0000","last_fullsized":"2024-04-17T07:57:22.981744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981978+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:57:22.981978+0000","last_peered":"2024-04-17T07:57:22.981978+0000","last_clean":"2024-04-17T07:57:22.981978+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:57:22.981978+0000","last_undegraded":"2024-04-17T07:57:22.981978+0000","last_fullsized":"2024-04-17T07:57:22.981978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298052+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:57:23.298052+0000","last_peered":"2024-04-17T07:57:23.298052+0000","last_clean":"2024-04-17T07:57:23.298052+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:57:23.298052+0000","last_undegraded":"2024-04-17T07:57:23.298052+0000","last_fullsized":"2024-04-17T07:57:23.298052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984677+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:57:22.984677+0000","last_peered":"2024-04-17T07:57:22.984677+0000","last_clean":"2024-04-17T07:57:22.984677+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:57:22.984677+0000","last_undegraded":"2024-04-17T07:57:22.984677+0000","last_fullsized":"2024-04-17T07:57:22.984677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986573+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:57:22.986573+0000","last_peered":"2024-04-17T07:57:22.986573+0000","last_clean":"2024-04-17T07:57:22.986573+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:57:22.986573+0000","last_undegraded":"2024-04-17T07:57:22.986573+0000","last_fullsized":"2024-04-17T07:57:22.986573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73231,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1035,"num_read_kb":598,"num_write":179,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":219693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383735,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1175946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808896,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":523921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070534,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":523841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332172,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1115207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495698,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":578095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":585350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1107287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:58:45.583 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-17T07:58:45.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:58:45.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:45 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/362198588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:45.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:45 smithi046 ceph-mon[32038]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:45.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:45 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/362198588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:45.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:45 smithi053 ceph-mon[36437]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:45.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:45 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/362198588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:45.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:45 smithi062 ceph-mon[36507]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 -- 172.21.15.46:0/395625653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105290 msgr2=0x7f4160105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 --2- 172.21.15.46:0/395625653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105290 0x7f4160105710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f414c009900 tx=0x7f414c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 -- 172.21.15.46:0/395625653 shutdown_connections 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 --2- 172.21.15.46:0/395625653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4160105c50 0x7f416010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 --2- 172.21.15.46:0/395625653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105290 0x7f4160105710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 --2- 172.21.15.46:0/395625653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41601040a0 0x7f41601044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 -- 172.21.15.46:0/395625653 >> 172.21.15.46:0/395625653 conn(0x7f41600ff630 msgr2=0x7f4160101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 -- 172.21.15.46:0/395625653 shutdown_connections 2024-04-17T07:58:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.838+0000 7f415f200640 1 -- 172.21.15.46:0/395625653 wait complete. 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.839+0000 7f415f200640 1 Processor -- start 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415f200640 1 -- start start 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41601040a0 0x7f416019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4160105290 0x7f416019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105c50 0x7f41601a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f416010fd80 con 0x7f41601040a0 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f416010fc00 con 0x7f4160105c50 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f416010ff00 con 0x7f4160105290 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41601040a0 0x7f416019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41601040a0 0x7f416019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60678/0 (socket says 172.21.15.46:60678) 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415de00640 1 -- 172.21.15.46:0/3407173761 learned_addr learned my addr 172.21.15.46:0/3407173761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415e800640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105c50 0x7f41601a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.840+0000 7f415d400640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4160105290 0x7f416019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415de00640 1 -- 172.21.15.46:0/3407173761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4160105290 msgr2=0x7f416019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415de00640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4160105290 0x7f416019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415de00640 1 -- 172.21.15.46:0/3407173761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105c50 msgr2=0x7f41601a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415de00640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105c50 0x7f41601a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415de00640 1 -- 172.21.15.46:0/3407173761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41601a1fb0 con 0x7f41601040a0 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415de00640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41601040a0 0x7f416019a390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f415000e9c0 tx=0x7f415000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f4156a00640 1 -- 172.21.15.46:0/3407173761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415000cd80 con 0x7f41601040a0 2024-04-17T07:58:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f4156a00640 1 -- 172.21.15.46:0/3407173761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41500035f0 con 0x7f41601040a0 2024-04-17T07:58:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f4156a00640 1 -- 172.21.15.46:0/3407173761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150010690 con 0x7f41601040a0 2024-04-17T07:58:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41601a22a0 con 0x7f41601040a0 2024-04-17T07:58:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.841+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41601a2770 con 0x7f41601040a0 2024-04-17T07:58:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.842+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416010b410 con 0x7f41601040a0 2024-04-17T07:58:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.845+0000 7f4156a00640 1 -- 172.21.15.46:0/3407173761 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f41500037a0 con 0x7f41601040a0 2024-04-17T07:58:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.846+0000 7f4156a00640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f41440a3ba0 0x7f41440a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.846+0000 7f4156a00640 1 -- 172.21.15.46:0/3407173761 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f41500d0bf0 con 0x7f41601040a0 2024-04-17T07:58:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.846+0000 7f415d400640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f41440a3ba0 0x7f41440a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.847+0000 7f415d400640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f41440a3ba0 0x7f41440a6060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f414c02c240 tx=0x7f414c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:45.851+0000 7f4156a00640 1 -- 172.21.15.46:0/3407173761 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f415009d1e0 con 0x7f41601040a0 2024-04-17T07:58:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.053+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 --> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41600623a0 con 0x7f41440a3ba0 2024-04-17T07:58:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.072+0000 7f4156a00640 1 -- 172.21.15.46:0/3407173761 <== mgr.14152 v2:172.21.15.46:6800/4172081855 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340496 (secure 0 0 0) 0x7f41600623a0 con 0x7f41440a3ba0 2024-04-17T07:58:46.073 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f41440a3ba0 msgr2=0x7f41440a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f41440a3ba0 0x7f41440a6060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f414c02c240 tx=0x7f414c02df90 comp rx=0 tx=0).stop 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41601040a0 msgr2=0x7f416019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41601040a0 0x7f416019a390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f415000e9c0 tx=0x7f415000ee80 comp rx=0 tx=0).stop 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 shutdown_connections 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f41440a3ba0 0x7f41440a6060 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4160105c50 0x7f41601a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4160105290 0x7f416019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 --2- 172.21.15.46:0/3407173761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41601040a0 0x7f416019a390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 >> 172.21.15.46:0/3407173761 conn(0x7f41600ff630 msgr2=0x7f41601085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 shutdown_connections 2024-04-17T07:58:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.080+0000 7f415f200640 1 -- 172.21.15.46:0/3407173761 wait complete. 2024-04-17T07:58:46.101 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-04-17T07:58:45.593203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663833,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1161,"num_read_kb":707,"num_write":362,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181973,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380340,"kb_used_data":14488,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561484,"statfs":{"total":1151940427776,"available":1151550959616,"internally_reserved":0,"allocated":14835712,"data_stored":8197124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5683,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":37,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004987"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987034+0000","last_change":"2024-04-17T07:55:26.359768+0000","last_active":"2024-04-17T07:57:22.987034+0000","last_peered":"2024-04-17T07:57:22.987034+0000","last_clean":"2024-04-17T07:57:22.987034+0000","last_became_active":"2024-04-17T07:55:26.359400+0000","last_became_peered":"2024-04-17T07:55:26.359400+0000","last_unstale":"2024-04-17T07:57:22.987034+0000","last_undegraded":"2024-04-17T07:57:22.987034+0000","last_fullsized":"2024-04-17T07:57:22.987034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:16.845734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301182+0000","last_change":"2024-04-17T07:55:26.366918+0000","last_active":"2024-04-17T07:57:23.301182+0000","last_peered":"2024-04-17T07:57:23.301182+0000","last_clean":"2024-04-17T07:57:23.301182+0000","last_became_active":"2024-04-17T07:55:26.362628+0000","last_became_peered":"2024-04-17T07:55:26.362628+0000","last_unstale":"2024-04-17T07:57:23.301182+0000","last_undegraded":"2024-04-17T07:57:23.301182+0000","last_fullsized":"2024-04-17T07:57:23.301182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:04.840740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300823+0000","last_change":"2024-04-17T07:55:26.366229+0000","last_active":"2024-04-17T07:57:23.300823+0000","last_peered":"2024-04-17T07:57:23.300823+0000","last_clean":"2024-04-17T07:57:23.300823+0000","last_became_active":"2024-04-17T07:55:26.364810+0000","last_became_peered":"2024-04-17T07:55:26.364810+0000","last_unstale":"2024-04-17T07:57:23.300823+0000","last_undegraded":"2024-04-17T07:57:23.300823+0000","last_fullsized":"2024-04-17T07:57:23.300823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:28.829550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987715+0000","last_change":"2024-04-17T07:55:26.353066+0000","last_active":"2024-04-17T07:57:22.987715+0000","last_peered":"2024-04-17T07:57:22.987715+0000","last_clean":"2024-04-17T07:57:22.987715+0000","last_became_active":"2024-04-17T07:55:26.352637+0000","last_became_peered":"2024-04-17T07:55:26.352637+0000","last_unstale":"2024-04-17T07:57:22.987715+0000","last_undegraded":"2024-04-17T07:57:22.987715+0000","last_fullsized":"2024-04-17T07:57:22.987715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:08:45.495744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989625+0000","last_change":"2024-04-17T07:55:26.374670+0000","last_active":"2024-04-17T07:57:22.989625+0000","last_peered":"2024-04-17T07:57:22.989625+0000","last_clean":"2024-04-17T07:57:22.989625+0000","last_became_active":"2024-04-17T07:55:26.374249+0000","last_became_peered":"2024-04-17T07:55:26.374249+0000","last_unstale":"2024-04-17T07:57:22.989625+0000","last_undegraded":"2024-04-17T07:57:22.989625+0000","last_fullsized":"2024-04-17T07:57:22.989625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:27.470533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990264+0000","last_change":"2024-04-17T07:55:26.367652+0000","last_active":"2024-04-17T07:57:22.990264+0000","last_peered":"2024-04-17T07:57:22.990264+0000","last_clean":"2024-04-17T07:57:22.990264+0000","last_became_active":"2024-04-17T07:55:26.367228+0000","last_became_peered":"2024-04-17T07:55:26.367228+0000","last_unstale":"2024-04-17T07:57:22.990264+0000","last_undegraded":"2024-04-17T07:57:22.990264+0000","last_fullsized":"2024-04-17T07:57:22.990264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:36.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989501+0000","last_change":"2024-04-17T07:55:26.366171+0000","last_active":"2024-04-17T07:57:22.989501+0000","last_peered":"2024-04-17T07:57:22.989501+0000","last_clean":"2024-04-17T07:57:22.989501+0000","last_became_active":"2024-04-17T07:55:26.365788+0000","last_became_peered":"2024-04-17T07:55:26.365788+0000","last_unstale":"2024-04-17T07:57:22.989501+0000","last_undegraded":"2024-04-17T07:57:22.989501+0000","last_fullsized":"2024-04-17T07:57:22.989501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:15.905494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981451+0000","last_change":"2024-04-17T07:55:26.371082+0000","last_active":"2024-04-17T07:57:22.981451+0000","last_peered":"2024-04-17T07:57:22.981451+0000","last_clean":"2024-04-17T07:57:22.981451+0000","last_became_active":"2024-04-17T07:55:26.366192+0000","last_became_peered":"2024-04-17T07:55:26.366192+0000","last_unstale":"2024-04-17T07:57:22.981451+0000","last_undegraded":"2024-04-17T07:57:22.981451+0000","last_fullsized":"2024-04-17T07:57:22.981451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:50.527780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298435+0000","last_change":"2024-04-17T07:55:26.349259+0000","last_active":"2024-04-17T07:57:23.298435+0000","last_peered":"2024-04-17T07:57:23.298435+0000","last_clean":"2024-04-17T07:57:23.298435+0000","last_became_active":"2024-04-17T07:55:26.348864+0000","last_became_peered":"2024-04-17T07:55:26.348864+0000","last_unstale":"2024-04-17T07:57:23.298435+0000","last_undegraded":"2024-04-17T07:57:23.298435+0000","last_fullsized":"2024-04-17T07:57:23.298435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:00.846900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991068+0000","last_change":"2024-04-17T07:55:26.351545+0000","last_active":"2024-04-17T07:57:22.991068+0000","last_peered":"2024-04-17T07:57:22.991068+0000","last_clean":"2024-04-17T07:57:22.991068+0000","last_became_active":"2024-04-17T07:55:26.351078+0000","last_became_peered":"2024-04-17T07:55:26.351078+0000","last_unstale":"2024-04-17T07:57:22.991068+0000","last_undegraded":"2024-04-17T07:57:22.991068+0000","last_fullsized":"2024-04-17T07:57:22.991068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:23.116717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990259+0000","last_change":"2024-04-17T07:55:26.348133+0000","last_active":"2024-04-17T07:57:22.990259+0000","last_peered":"2024-04-17T07:57:22.990259+0000","last_clean":"2024-04-17T07:57:22.990259+0000","last_became_active":"2024-04-17T07:55:26.347499+0000","last_became_peered":"2024-04-17T07:55:26.347499+0000","last_unstale":"2024-04-17T07:57:22.990259+0000","last_undegraded":"2024-04-17T07:57:22.990259+0000","last_fullsized":"2024-04-17T07:57:22.990259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:02.927592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987830+0000","last_change":"2024-04-17T07:55:26.351354+0000","last_active":"2024-04-17T07:57:22.987830+0000","last_peered":"2024-04-17T07:57:22.987830+0000","last_clean":"2024-04-17T07:57:22.987830+0000","last_became_active":"2024-04-17T07:55:26.350857+0000","last_became_peered":"2024-04-17T07:55:26.350857+0000","last_unstale":"2024-04-17T07:57:22.987830+0000","last_undegraded":"2024-04-17T07:57:22.987830+0000","last_fullsized":"2024-04-17T07:57:22.987830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:42.021103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984491+0000","last_change":"2024-04-17T07:55:26.355098+0000","last_active":"2024-04-17T07:57:22.984491+0000","last_peered":"2024-04-17T07:57:22.984491+0000","last_clean":"2024-04-17T07:57:22.984491+0000","last_became_active":"2024-04-17T07:55:26.351835+0000","last_became_peered":"2024-04-17T07:55:26.351835+0000","last_unstale":"2024-04-17T07:57:22.984491+0000","last_undegraded":"2024-04-17T07:57:22.984491+0000","last_fullsized":"2024-04-17T07:57:22.984491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:52.314790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300180+0000","last_change":"2024-04-17T07:55:26.359705+0000","last_active":"2024-04-17T07:57:23.300180+0000","last_peered":"2024-04-17T07:57:23.300180+0000","last_clean":"2024-04-17T07:57:23.300180+0000","last_became_active":"2024-04-17T07:55:26.359284+0000","last_became_peered":"2024-04-17T07:55:26.359284+0000","last_unstale":"2024-04-17T07:57:23.300180+0000","last_undegraded":"2024-04-17T07:57:23.300180+0000","last_fullsized":"2024-04-17T07:57:23.300180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:45:21.638597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300300+0000","last_change":"2024-04-17T07:55:26.361480+0000","last_active":"2024-04-17T07:57:23.300300+0000","last_peered":"2024-04-17T07:57:23.300300+0000","last_clean":"2024-04-17T07:57:23.300300+0000","last_became_active":"2024-04-17T07:55:26.360402+0000","last_became_peered":"2024-04-17T07:55:26.360402+0000","last_unstale":"2024-04-17T07:57:23.300300+0000","last_undegraded":"2024-04-17T07:57:23.300300+0000","last_fullsized":"2024-04-17T07:57:23.300300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:59.892787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297670+0000","last_change":"2024-04-17T07:55:26.364068+0000","last_active":"2024-04-17T07:57:23.297670+0000","last_peered":"2024-04-17T07:57:23.297670+0000","last_clean":"2024-04-17T07:57:23.297670+0000","last_became_active":"2024-04-17T07:55:26.363710+0000","last_became_peered":"2024-04-17T07:55:26.363710+0000","last_unstale":"2024-04-17T07:57:23.297670+0000","last_undegraded":"2024-04-17T07:57:23.297670+0000","last_fullsized":"2024-04-17T07:57:23.297670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:49.059450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986912+0000","last_change":"2024-04-17T07:55:26.359725+0000","last_active":"2024-04-17T07:57:22.986912+0000","last_peered":"2024-04-17T07:57:22.986912+0000","last_clean":"2024-04-17T07:57:22.986912+0000","last_became_active":"2024-04-17T07:55:26.359321+0000","last_became_peered":"2024-04-17T07:55:26.359321+0000","last_unstale":"2024-04-17T07:57:22.986912+0000","last_undegraded":"2024-04-17T07:57:22.986912+0000","last_fullsized":"2024-04-17T07:57:22.986912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:54:46.247493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299088+0000","last_change":"2024-04-17T07:55:26.364698+0000","last_active":"2024-04-17T07:57:23.299088+0000","last_peered":"2024-04-17T07:57:23.299088+0000","last_clean":"2024-04-17T07:57:23.299088+0000","last_became_active":"2024-04-17T07:55:26.364310+0000","last_became_peered":"2024-04-17T07:55:26.364310+0000","last_unstale":"2024-04-17T07:57:23.299088+0000","last_undegraded":"2024-04-17T07:57:23.299088+0000","last_fullsized":"2024-04-17T07:57:23.299088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:13:31.477907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990554+0000","last_change":"2024-04-17T07:55:26.362342+0000","last_active":"2024-04-17T07:57:22.990554+0000","last_peered":"2024-04-17T07:57:22.990554+0000","last_clean":"2024-04-17T07:57:22.990554+0000","last_became_active":"2024-04-17T07:55:26.361941+0000","last_became_peered":"2024-04-17T07:55:26.361941+0000","last_unstale":"2024-04-17T07:57:22.990554+0000","last_undegraded":"2024-04-17T07:57:22.990554+0000","last_fullsized":"2024-04-17T07:57:22.990554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:14:51.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990152+0000","last_change":"2024-04-17T07:55:30.600623+0000","last_active":"2024-04-17T07:57:22.990152+0000","last_peered":"2024-04-17T07:57:22.990152+0000","last_clean":"2024-04-17T07:57:22.990152+0000","last_became_active":"2024-04-17T07:55:30.600156+0000","last_became_peered":"2024-04-17T07:55:30.600156+0000","last_unstale":"2024-04-17T07:57:22.990152+0000","last_undegraded":"2024-04-17T07:57:22.990152+0000","last_fullsized":"2024-04-17T07:57:22.990152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:04:55.459736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299637+0000","last_change":"2024-04-17T07:55:25.520640+0000","last_active":"2024-04-17T07:57:23.299637+0000","last_peered":"2024-04-17T07:57:23.299637+0000","last_clean":"2024-04-17T07:57:23.299637+0000","last_became_active":"2024-04-17T07:55:25.520213+0000","last_became_peered":"2024-04-17T07:55:25.520213+0000","last_unstale":"2024-04-17T07:57:23.299637+0000","last_undegraded":"2024-04-17T07:57:23.299637+0000","last_fullsized":"2024-04-17T07:57:23.299637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:13.981651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298966+0000","last_change":"2024-04-17T07:56:22.833540+0000","last_active":"2024-04-17T07:57:23.298966+0000","last_peered":"2024-04-17T07:57:23.298966+0000","last_clean":"2024-04-17T07:57:23.298966+0000","last_became_active":"2024-04-17T07:56:22.832620+0000","last_became_peered":"2024-04-17T07:56:22.832620+0000","last_unstale":"2024-04-17T07:57:23.298966+0000","last_undegraded":"2024-04-17T07:57:23.298966+0000","last_fullsized":"2024-04-17T07:57:23.298966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:14:42.827111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987127+0000","last_change":"2024-04-17T07:55:30.599860+0000","last_active":"2024-04-17T07:57:22.987127+0000","last_peered":"2024-04-17T07:57:22.987127+0000","last_clean":"2024-04-17T07:57:22.987127+0000","last_became_active":"2024-04-17T07:55:30.599320+0000","last_became_peered":"2024-04-17T07:55:30.599320+0000","last_unstale":"2024-04-17T07:57:22.987127+0000","last_undegraded":"2024-04-17T07:57:22.987127+0000","last_fullsized":"2024-04-17T07:57:22.987127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.857471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984617+0000","last_change":"2024-04-17T07:55:25.539749+0000","last_active":"2024-04-17T07:57:22.984617+0000","last_peered":"2024-04-17T07:57:22.984617+0000","last_clean":"2024-04-17T07:57:22.984617+0000","last_became_active":"2024-04-17T07:55:25.537176+0000","last_became_peered":"2024-04-17T07:55:25.537176+0000","last_unstale":"2024-04-17T07:57:22.984617+0000","last_undegraded":"2024-04-17T07:57:22.984617+0000","last_fullsized":"2024-04-17T07:57:22.984617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:44:01.766031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301222+0000","last_change":"2024-04-17T07:55:26.368787+0000","last_active":"2024-04-17T07:57:23.301222+0000","last_peered":"2024-04-17T07:57:23.301222+0000","last_clean":"2024-04-17T07:57:23.301222+0000","last_became_active":"2024-04-17T07:55:26.368432+0000","last_became_peered":"2024-04-17T07:55:26.368432+0000","last_unstale":"2024-04-17T07:57:23.301222+0000","last_undegraded":"2024-04-17T07:57:23.301222+0000","last_fullsized":"2024-04-17T07:57:23.301222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:06.107609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987203+0000","last_change":"2024-04-17T07:55:30.605537+0000","last_active":"2024-04-17T07:57:22.987203+0000","last_peered":"2024-04-17T07:57:22.987203+0000","last_clean":"2024-04-17T07:57:22.987203+0000","last_became_active":"2024-04-17T07:55:30.605179+0000","last_became_peered":"2024-04-17T07:55:30.605179+0000","last_unstale":"2024-04-17T07:57:22.987203+0000","last_undegraded":"2024-04-17T07:57:22.987203+0000","last_fullsized":"2024-04-17T07:57:22.987203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:08:35.638269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.996476+0000","last_change":"2024-04-17T07:57:22.996476+0000","last_active":"2024-04-17T07:57:22.996476+0000","last_peered":"2024-04-17T07:57:22.996476+0000","last_clean":"2024-04-17T07:57:22.996476+0000","last_became_active":"2024-04-17T07:57:22.991654+0000","last_became_peered":"2024-04-17T07:57:22.991654+0000","last_unstale":"2024-04-17T07:57:22.996476+0000","last_undegraded":"2024-04-17T07:57:22.996476+0000","last_fullsized":"2024-04-17T07:57:22.996476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:33.704606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298058+0000","last_change":"2024-04-17T07:55:26.355474+0000","last_active":"2024-04-17T07:57:23.298058+0000","last_peered":"2024-04-17T07:57:23.298058+0000","last_clean":"2024-04-17T07:57:23.298058+0000","last_became_active":"2024-04-17T07:55:26.355094+0000","last_became_peered":"2024-04-17T07:55:26.355094+0000","last_unstale":"2024-04-17T07:57:23.298058+0000","last_undegraded":"2024-04-17T07:57:23.298058+0000","last_fullsized":"2024-04-17T07:57:23.298058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:59.394266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987151+0000","last_change":"2024-04-17T07:55:30.606991+0000","last_active":"2024-04-17T07:57:22.987151+0000","last_peered":"2024-04-17T07:57:22.987151+0000","last_clean":"2024-04-17T07:57:22.987151+0000","last_became_active":"2024-04-17T07:55:30.606487+0000","last_became_peered":"2024-04-17T07:55:30.606487+0000","last_unstale":"2024-04-17T07:57:22.987151+0000","last_undegraded":"2024-04-17T07:57:22.987151+0000","last_fullsized":"2024-04-17T07:57:22.987151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:34:32.165371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990261+0000","last_change":"2024-04-17T07:55:25.508179+0000","last_active":"2024-04-17T07:57:22.990261+0000","last_peered":"2024-04-17T07:57:22.990261+0000","last_clean":"2024-04-17T07:57:22.990261+0000","last_became_active":"2024-04-17T07:55:25.507848+0000","last_became_peered":"2024-04-17T07:55:25.507848+0000","last_unstale":"2024-04-17T07:57:22.990261+0000","last_undegraded":"2024-04-17T07:57:22.990261+0000","last_fullsized":"2024-04-17T07:57:22.990261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:48:03.349274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982016+0000","last_change":"2024-04-17T07:55:26.345307+0000","last_active":"2024-04-17T07:57:22.982016+0000","last_peered":"2024-04-17T07:57:22.982016+0000","last_clean":"2024-04-17T07:57:22.982016+0000","last_became_active":"2024-04-17T07:55:26.344968+0000","last_became_peered":"2024-04-17T07:55:26.344968+0000","last_unstale":"2024-04-17T07:57:22.982016+0000","last_undegraded":"2024-04-17T07:57:22.982016+0000","last_fullsized":"2024-04-17T07:57:22.982016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:10.865306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990411+0000","last_change":"2024-04-17T07:55:30.592027+0000","last_active":"2024-04-17T07:57:22.990411+0000","last_peered":"2024-04-17T07:57:22.990411+0000","last_clean":"2024-04-17T07:57:22.990411+0000","last_became_active":"2024-04-17T07:55:30.591288+0000","last_became_peered":"2024-04-17T07:55:30.591288+0000","last_unstale":"2024-04-17T07:57:22.990411+0000","last_undegraded":"2024-04-17T07:57:22.990411+0000","last_fullsized":"2024-04-17T07:57:22.990411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:53.919489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":210,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:39.804590+0000","last_change":"2024-04-17T07:55:25.509640+0000","last_active":"2024-04-17T07:58:39.804590+0000","last_peered":"2024-04-17T07:58:39.804590+0000","last_clean":"2024-04-17T07:58:39.804590+0000","last_became_active":"2024-04-17T07:55:25.509297+0000","last_became_peered":"2024-04-17T07:55:25.509297+0000","last_unstale":"2024-04-17T07:58:39.804590+0000","last_undegraded":"2024-04-17T07:58:39.804590+0000","last_fullsized":"2024-04-17T07:58:39.804590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:49:21.572486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990055+0000","last_change":"2024-04-17T07:55:26.368230+0000","last_active":"2024-04-17T07:57:22.990055+0000","last_peered":"2024-04-17T07:57:22.990055+0000","last_clean":"2024-04-17T07:57:22.990055+0000","last_became_active":"2024-04-17T07:55:26.367832+0000","last_became_peered":"2024-04-17T07:55:26.367832+0000","last_unstale":"2024-04-17T07:57:22.990055+0000","last_undegraded":"2024-04-17T07:57:22.990055+0000","last_fullsized":"2024-04-17T07:57:22.990055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:02:22.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989753+0000","last_change":"2024-04-17T07:55:30.606739+0000","last_active":"2024-04-17T07:57:22.989753+0000","last_peered":"2024-04-17T07:57:22.989753+0000","last_clean":"2024-04-17T07:57:22.989753+0000","last_became_active":"2024-04-17T07:55:30.606285+0000","last_became_peered":"2024-04-17T07:55:30.606285+0000","last_unstale":"2024-04-17T07:57:22.989753+0000","last_undegraded":"2024-04-17T07:57:22.989753+0000","last_fullsized":"2024-04-17T07:57:22.989753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:51.486988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990141+0000","last_change":"2024-04-17T07:55:25.504267+0000","last_active":"2024-04-17T07:57:22.990141+0000","last_peered":"2024-04-17T07:57:22.990141+0000","last_clean":"2024-04-17T07:57:22.990141+0000","last_became_active":"2024-04-17T07:55:25.503897+0000","last_became_peered":"2024-04-17T07:55:25.503897+0000","last_unstale":"2024-04-17T07:57:22.990141+0000","last_undegraded":"2024-04-17T07:57:22.990141+0000","last_fullsized":"2024-04-17T07:57:22.990141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.518625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.366469+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.366147+0000","last_became_peered":"2024-04-17T07:55:26.366147+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:37.700404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986902+0000","last_change":"2024-04-17T07:55:30.589795+0000","last_active":"2024-04-17T07:57:22.986902+0000","last_peered":"2024-04-17T07:57:22.986902+0000","last_clean":"2024-04-17T07:57:22.986902+0000","last_became_active":"2024-04-17T07:55:30.589313+0000","last_became_peered":"2024-04-17T07:55:30.589313+0000","last_unstale":"2024-04-17T07:57:22.986902+0000","last_undegraded":"2024-04-17T07:57:22.986902+0000","last_fullsized":"2024-04-17T07:57:22.986902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:07.802965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982901+0000","last_change":"2024-04-17T07:55:25.521964+0000","last_active":"2024-04-17T07:57:22.982901+0000","last_peered":"2024-04-17T07:57:22.982901+0000","last_clean":"2024-04-17T07:57:22.982901+0000","last_became_active":"2024-04-17T07:55:25.520363+0000","last_became_peered":"2024-04-17T07:55:25.520363+0000","last_unstale":"2024-04-17T07:57:22.982901+0000","last_undegraded":"2024-04-17T07:57:22.982901+0000","last_fullsized":"2024-04-17T07:57:22.982901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:50.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981110+0000","last_change":"2024-04-17T07:55:26.375983+0000","last_active":"2024-04-17T07:57:22.981110+0000","last_peered":"2024-04-17T07:57:22.981110+0000","last_clean":"2024-04-17T07:57:22.981110+0000","last_became_active":"2024-04-17T07:55:26.374974+0000","last_became_peered":"2024-04-17T07:55:26.374974+0000","last_unstale":"2024-04-17T07:57:22.981110+0000","last_undegraded":"2024-04-17T07:57:22.981110+0000","last_fullsized":"2024-04-17T07:57:22.981110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:24.952062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301002+0000","last_change":"2024-04-17T07:55:30.608215+0000","last_active":"2024-04-17T07:57:23.301002+0000","last_peered":"2024-04-17T07:57:23.301002+0000","last_clean":"2024-04-17T07:57:23.301002+0000","last_became_active":"2024-04-17T07:55:30.604883+0000","last_became_peered":"2024-04-17T07:55:30.604883+0000","last_unstale":"2024-04-17T07:57:23.301002+0000","last_undegraded":"2024-04-17T07:57:23.301002+0000","last_fullsized":"2024-04-17T07:57:23.301002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:38.002460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981473+0000","last_change":"2024-04-17T07:55:25.507368+0000","last_active":"2024-04-17T07:57:22.981473+0000","last_peered":"2024-04-17T07:57:22.981473+0000","last_clean":"2024-04-17T07:57:22.981473+0000","last_became_active":"2024-04-17T07:55:25.507031+0000","last_became_peered":"2024-04-17T07:55:25.507031+0000","last_unstale":"2024-04-17T07:57:22.981473+0000","last_undegraded":"2024-04-17T07:57:22.981473+0000","last_fullsized":"2024-04-17T07:57:22.981473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:00:47.695144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299094+0000","last_change":"2024-04-17T07:55:26.375054+0000","last_active":"2024-04-17T07:57:23.299094+0000","last_peered":"2024-04-17T07:57:23.299094+0000","last_clean":"2024-04-17T07:57:23.299094+0000","last_became_active":"2024-04-17T07:55:26.374657+0000","last_became_peered":"2024-04-17T07:55:26.374657+0000","last_unstale":"2024-04-17T07:57:23.299094+0000","last_undegraded":"2024-04-17T07:57:23.299094+0000","last_fullsized":"2024-04-17T07:57:23.299094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:11.193833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981348+0000","last_change":"2024-04-17T07:55:30.600918+0000","last_active":"2024-04-17T07:57:22.981348+0000","last_peered":"2024-04-17T07:57:22.981348+0000","last_clean":"2024-04-17T07:57:22.981348+0000","last_became_active":"2024-04-17T07:55:30.600277+0000","last_became_peered":"2024-04-17T07:55:30.600277+0000","last_unstale":"2024-04-17T07:57:22.981348+0000","last_undegraded":"2024-04-17T07:57:22.981348+0000","last_fullsized":"2024-04-17T07:57:22.981348+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:34.294513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":204,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:40.805698+0000","last_change":"2024-04-17T07:55:25.519836+0000","last_active":"2024-04-17T07:58:40.805698+0000","last_peered":"2024-04-17T07:58:40.805698+0000","last_clean":"2024-04-17T07:58:40.805698+0000","last_became_active":"2024-04-17T07:55:25.519574+0000","last_became_peered":"2024-04-17T07:55:25.519574+0000","last_unstale":"2024-04-17T07:58:40.805698+0000","last_undegraded":"2024-04-17T07:58:40.805698+0000","last_fullsized":"2024-04-17T07:58:40.805698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:41:39.460867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298436+0000","last_change":"2024-04-17T07:55:26.368675+0000","last_active":"2024-04-17T07:57:23.298436+0000","last_peered":"2024-04-17T07:57:23.298436+0000","last_clean":"2024-04-17T07:57:23.298436+0000","last_became_active":"2024-04-17T07:55:26.368056+0000","last_became_peered":"2024-04-17T07:55:26.368056+0000","last_unstale":"2024-04-17T07:57:23.298436+0000","last_undegraded":"2024-04-17T07:57:23.298436+0000","last_fullsized":"2024-04-17T07:57:23.298436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:54:31.531570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299555+0000","last_change":"2024-04-17T07:55:30.592113+0000","last_active":"2024-04-17T07:57:23.299555+0000","last_peered":"2024-04-17T07:57:23.299555+0000","last_clean":"2024-04-17T07:57:23.299555+0000","last_became_active":"2024-04-17T07:55:30.591657+0000","last_became_peered":"2024-04-17T07:55:30.591657+0000","last_unstale":"2024-04-17T07:57:23.299555+0000","last_undegraded":"2024-04-17T07:57:23.299555+0000","last_fullsized":"2024-04-17T07:57:23.299555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:21:22.855769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990114+0000","last_change":"2024-04-17T07:55:25.505487+0000","last_active":"2024-04-17T07:57:22.990114+0000","last_peered":"2024-04-17T07:57:22.990114+0000","last_clean":"2024-04-17T07:57:22.990114+0000","last_became_active":"2024-04-17T07:55:25.504829+0000","last_became_peered":"2024-04-17T07:55:25.504829+0000","last_unstale":"2024-04-17T07:57:22.990114+0000","last_undegraded":"2024-04-17T07:57:22.990114+0000","last_fullsized":"2024-04-17T07:57:22.990114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:33.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301169+0000","last_change":"2024-04-17T07:55:26.361006+0000","last_active":"2024-04-17T07:57:23.301169+0000","last_peered":"2024-04-17T07:57:23.301169+0000","last_clean":"2024-04-17T07:57:23.301169+0000","last_became_active":"2024-04-17T07:55:26.359254+0000","last_became_peered":"2024-04-17T07:55:26.359254+0000","last_unstale":"2024-04-17T07:57:23.301169+0000","last_undegraded":"2024-04-17T07:57:23.301169+0000","last_fullsized":"2024-04-17T07:57:23.301169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:27:02.586166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987333+0000","last_change":"2024-04-17T07:55:30.592554+0000","last_active":"2024-04-17T07:57:22.987333+0000","last_peered":"2024-04-17T07:57:22.987333+0000","last_clean":"2024-04-17T07:57:22.987333+0000","last_became_active":"2024-04-17T07:55:30.591849+0000","last_became_peered":"2024-04-17T07:55:30.591849+0000","last_unstale":"2024-04-17T07:57:22.987333+0000","last_undegraded":"2024-04-17T07:57:22.987333+0000","last_fullsized":"2024-04-17T07:57:22.987333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:46.537086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981989+0000","last_change":"2024-04-17T07:55:25.533189+0000","last_active":"2024-04-17T07:57:22.981989+0000","last_peered":"2024-04-17T07:57:22.981989+0000","last_clean":"2024-04-17T07:57:22.981989+0000","last_became_active":"2024-04-17T07:55:25.532928+0000","last_became_peered":"2024-04-17T07:55:25.532928+0000","last_unstale":"2024-04-17T07:57:22.981989+0000","last_undegraded":"2024-04-17T07:57:22.981989+0000","last_fullsized":"2024-04-17T07:57:22.981989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:03.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298180+0000","last_change":"2024-04-17T07:55:26.354620+0000","last_active":"2024-04-17T07:57:23.298180+0000","last_peered":"2024-04-17T07:57:23.298180+0000","last_clean":"2024-04-17T07:57:23.298180+0000","last_became_active":"2024-04-17T07:55:26.352558+0000","last_became_peered":"2024-04-17T07:55:26.352558+0000","last_unstale":"2024-04-17T07:57:23.298180+0000","last_undegraded":"2024-04-17T07:57:23.298180+0000","last_fullsized":"2024-04-17T07:57:23.298180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:02:09.352109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298705+0000","last_change":"2024-04-17T07:55:30.586853+0000","last_active":"2024-04-17T07:57:23.298705+0000","last_peered":"2024-04-17T07:57:23.298705+0000","last_clean":"2024-04-17T07:57:23.298705+0000","last_became_active":"2024-04-17T07:55:30.586040+0000","last_became_peered":"2024-04-17T07:55:30.586040+0000","last_unstale":"2024-04-17T07:57:23.298705+0000","last_undegraded":"2024-04-17T07:57:23.298705+0000","last_fullsized":"2024-04-17T07:57:23.298705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.365801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":417,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:41.808235+0000","last_change":"2024-04-17T07:55:25.505239+0000","last_active":"2024-04-17T07:58:41.808235+0000","last_peered":"2024-04-17T07:58:41.808235+0000","last_clean":"2024-04-17T07:58:41.808235+0000","last_became_active":"2024-04-17T07:55:25.504886+0000","last_became_peered":"2024-04-17T07:55:25.504886+0000","last_unstale":"2024-04-17T07:58:41.808235+0000","last_undegraded":"2024-04-17T07:58:41.808235+0000","last_fullsized":"2024-04-17T07:58:41.808235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:45.136410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68557,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":347,"num_read_kb":242,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991109+0000","last_change":"2024-04-17T07:55:26.345861+0000","last_active":"2024-04-17T07:57:22.991109+0000","last_peered":"2024-04-17T07:57:22.991109+0000","last_clean":"2024-04-17T07:57:22.991109+0000","last_became_active":"2024-04-17T07:55:26.345504+0000","last_became_peered":"2024-04-17T07:55:26.345504+0000","last_unstale":"2024-04-17T07:57:22.991109+0000","last_undegraded":"2024-04-17T07:57:22.991109+0000","last_fullsized":"2024-04-17T07:57:22.991109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:32.122977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982223+0000","last_change":"2024-04-17T07:55:30.602152+0000","last_active":"2024-04-17T07:57:22.982223+0000","last_peered":"2024-04-17T07:57:22.982223+0000","last_clean":"2024-04-17T07:57:22.982223+0000","last_became_active":"2024-04-17T07:55:30.600932+0000","last_became_peered":"2024-04-17T07:55:30.600932+0000","last_unstale":"2024-04-17T07:57:22.982223+0000","last_undegraded":"2024-04-17T07:57:22.982223+0000","last_fullsized":"2024-04-17T07:57:22.982223+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:57:22.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989456+0000","last_change":"2024-04-17T07:55:25.536114+0000","last_active":"2024-04-17T07:57:22.989456+0000","last_peered":"2024-04-17T07:57:22.989456+0000","last_clean":"2024-04-17T07:57:22.989456+0000","last_became_active":"2024-04-17T07:55:25.535790+0000","last_became_peered":"2024-04-17T07:55:25.535790+0000","last_unstale":"2024-04-17T07:57:22.989456+0000","last_undegraded":"2024-04-17T07:57:22.989456+0000","last_fullsized":"2024-04-17T07:57:22.989456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:30.644829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983461+0000","last_change":"2024-04-17T07:55:26.375018+0000","last_active":"2024-04-17T07:57:22.983461+0000","last_peered":"2024-04-17T07:57:22.983461+0000","last_clean":"2024-04-17T07:57:22.983461+0000","last_became_active":"2024-04-17T07:55:26.373166+0000","last_became_peered":"2024-04-17T07:55:26.373166+0000","last_unstale":"2024-04-17T07:57:22.983461+0000","last_undegraded":"2024-04-17T07:57:22.983461+0000","last_fullsized":"2024-04-17T07:57:22.983461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:29:29.557772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986452+0000","last_change":"2024-04-17T07:55:30.601196+0000","last_active":"2024-04-17T07:57:22.986452+0000","last_peered":"2024-04-17T07:57:22.986452+0000","last_clean":"2024-04-17T07:57:22.986452+0000","last_became_active":"2024-04-17T07:55:30.600755+0000","last_became_peered":"2024-04-17T07:55:30.600755+0000","last_unstale":"2024-04-17T07:57:22.986452+0000","last_undegraded":"2024-04-17T07:57:22.986452+0000","last_fullsized":"2024-04-17T07:57:22.986452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:13.491333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981873+0000","last_change":"2024-04-17T07:55:25.537800+0000","last_active":"2024-04-17T07:57:22.981873+0000","last_peered":"2024-04-17T07:57:22.981873+0000","last_clean":"2024-04-17T07:57:22.981873+0000","last_became_active":"2024-04-17T07:55:25.537427+0000","last_became_peered":"2024-04-17T07:55:25.537427+0000","last_unstale":"2024-04-17T07:57:22.981873+0000","last_undegraded":"2024-04-17T07:57:22.981873+0000","last_fullsized":"2024-04-17T07:57:22.981873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:58.334219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989629+0000","last_change":"2024-04-17T07:56:22.834205+0000","last_active":"2024-04-17T07:57:22.989629+0000","last_peered":"2024-04-17T07:57:22.989629+0000","last_clean":"2024-04-17T07:57:22.989629+0000","last_became_active":"2024-04-17T07:56:22.833763+0000","last_became_peered":"2024-04-17T07:56:22.833763+0000","last_unstale":"2024-04-17T07:57:22.989629+0000","last_undegraded":"2024-04-17T07:57:22.989629+0000","last_fullsized":"2024-04-17T07:57:22.989629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:04:24.761128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299422+0000","last_change":"2024-04-17T07:55:30.614781+0000","last_active":"2024-04-17T07:57:23.299422+0000","last_peered":"2024-04-17T07:57:23.299422+0000","last_clean":"2024-04-17T07:57:23.299422+0000","last_became_active":"2024-04-17T07:55:30.614349+0000","last_became_peered":"2024-04-17T07:55:30.614349+0000","last_unstale":"2024-04-17T07:57:23.299422+0000","last_undegraded":"2024-04-17T07:57:23.299422+0000","last_fullsized":"2024-04-17T07:57:23.299422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:20.937592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":219,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:39.166397+0000","last_change":"2024-04-17T07:57:22.995880+0000","last_active":"2024-04-17T07:58:39.166397+0000","last_peered":"2024-04-17T07:58:39.166397+0000","last_clean":"2024-04-17T07:58:39.166397+0000","last_became_active":"2024-04-17T07:57:22.991967+0000","last_became_peered":"2024-04-17T07:57:22.991967+0000","last_unstale":"2024-04-17T07:58:39.166397+0000","last_undegraded":"2024-04-17T07:58:39.166397+0000","last_fullsized":"2024-04-17T07:58:39.166397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:27.207869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989564+0000","last_change":"2024-04-17T07:55:26.345002+0000","last_active":"2024-04-17T07:57:22.989564+0000","last_peered":"2024-04-17T07:57:22.989564+0000","last_clean":"2024-04-17T07:57:22.989564+0000","last_became_active":"2024-04-17T07:55:26.344657+0000","last_became_peered":"2024-04-17T07:55:26.344657+0000","last_unstale":"2024-04-17T07:57:22.989564+0000","last_undegraded":"2024-04-17T07:57:22.989564+0000","last_fullsized":"2024-04-17T07:57:22.989564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:06:10.330444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984648+0000","last_change":"2024-04-17T07:56:22.837796+0000","last_active":"2024-04-17T07:57:22.984648+0000","last_peered":"2024-04-17T07:57:22.984648+0000","last_clean":"2024-04-17T07:57:22.984648+0000","last_became_active":"2024-04-17T07:56:22.837051+0000","last_became_peered":"2024-04-17T07:56:22.837051+0000","last_unstale":"2024-04-17T07:57:22.984648+0000","last_undegraded":"2024-04-17T07:57:22.984648+0000","last_fullsized":"2024-04-17T07:57:22.984648+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:20:21.157412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989830+0000","last_change":"2024-04-17T07:55:25.535798+0000","last_active":"2024-04-17T07:57:22.989830+0000","last_peered":"2024-04-17T07:57:22.989830+0000","last_clean":"2024-04-17T07:57:22.989830+0000","last_became_active":"2024-04-17T07:55:25.535456+0000","last_became_peered":"2024-04-17T07:55:25.535456+0000","last_unstale":"2024-04-17T07:57:22.989830+0000","last_undegraded":"2024-04-17T07:57:22.989830+0000","last_fullsized":"2024-04-17T07:57:22.989830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:38.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981593+0000","last_change":"2024-04-17T07:55:26.346418+0000","last_active":"2024-04-17T07:57:22.981593+0000","last_peered":"2024-04-17T07:57:22.981593+0000","last_clean":"2024-04-17T07:57:22.981593+0000","last_became_active":"2024-04-17T07:55:26.346036+0000","last_became_peered":"2024-04-17T07:55:26.346036+0000","last_unstale":"2024-04-17T07:57:22.981593+0000","last_undegraded":"2024-04-17T07:57:22.981593+0000","last_fullsized":"2024-04-17T07:57:22.981593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:49.980130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981392+0000","last_change":"2024-04-17T07:55:30.600600+0000","last_active":"2024-04-17T07:57:22.981392+0000","last_peered":"2024-04-17T07:57:22.981392+0000","last_clean":"2024-04-17T07:57:22.981392+0000","last_became_active":"2024-04-17T07:55:30.600080+0000","last_became_peered":"2024-04-17T07:55:30.600080+0000","last_unstale":"2024-04-17T07:57:22.981392+0000","last_undegraded":"2024-04-17T07:57:22.981392+0000","last_fullsized":"2024-04-17T07:57:22.981392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:02:26.683067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991262+0000","last_change":"2024-04-17T07:55:25.508262+0000","last_active":"2024-04-17T07:57:22.991262+0000","last_peered":"2024-04-17T07:57:22.991262+0000","last_clean":"2024-04-17T07:57:22.991262+0000","last_became_active":"2024-04-17T07:55:25.507935+0000","last_became_peered":"2024-04-17T07:55:25.507935+0000","last_unstale":"2024-04-17T07:57:22.991262+0000","last_undegraded":"2024-04-17T07:57:22.991262+0000","last_fullsized":"2024-04-17T07:57:22.991262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:49:14.319308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986868+0000","last_change":"2024-04-17T07:55:26.353400+0000","last_active":"2024-04-17T07:57:22.986868+0000","last_peered":"2024-04-17T07:57:22.986868+0000","last_clean":"2024-04-17T07:57:22.986868+0000","last_became_active":"2024-04-17T07:55:26.353034+0000","last_became_peered":"2024-04-17T07:55:26.353034+0000","last_unstale":"2024-04-17T07:57:22.986868+0000","last_undegraded":"2024-04-17T07:57:22.986868+0000","last_fullsized":"2024-04-17T07:57:22.986868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:44.530090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989693+0000","last_change":"2024-04-17T07:56:23.583770+0000","last_active":"2024-04-17T07:57:22.989693+0000","last_peered":"2024-04-17T07:57:22.989693+0000","last_clean":"2024-04-17T07:57:22.989693+0000","last_became_active":"2024-04-17T07:56:23.583309+0000","last_became_peered":"2024-04-17T07:56:23.583309+0000","last_unstale":"2024-04-17T07:57:22.989693+0000","last_undegraded":"2024-04-17T07:57:22.989693+0000","last_fullsized":"2024-04-17T07:57:22.989693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:59.014143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299647+0000","last_change":"2024-04-17T07:55:25.535870+0000","last_active":"2024-04-17T07:57:23.299647+0000","last_peered":"2024-04-17T07:57:23.299647+0000","last_clean":"2024-04-17T07:57:23.299647+0000","last_became_active":"2024-04-17T07:55:25.535445+0000","last_became_peered":"2024-04-17T07:55:25.535445+0000","last_unstale":"2024-04-17T07:57:23.299647+0000","last_undegraded":"2024-04-17T07:57:23.299647+0000","last_fullsized":"2024-04-17T07:57:23.299647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:16.964865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:26.370787+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:26.370473+0000","last_became_peered":"2024-04-17T07:55:26.370473+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:05.736437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982610+0000","last_change":"2024-04-17T07:55:30.611072+0000","last_active":"2024-04-17T07:57:22.982610+0000","last_peered":"2024-04-17T07:57:22.982610+0000","last_clean":"2024-04-17T07:57:22.982610+0000","last_became_active":"2024-04-17T07:55:30.610586+0000","last_became_peered":"2024-04-17T07:55:30.610586+0000","last_unstale":"2024-04-17T07:57:22.982610+0000","last_undegraded":"2024-04-17T07:57:22.982610+0000","last_fullsized":"2024-04-17T07:57:22.982610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:46.284555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989416+0000","last_change":"2024-04-17T07:55:25.507883+0000","last_active":"2024-04-17T07:57:22.989416+0000","last_peered":"2024-04-17T07:57:22.989416+0000","last_clean":"2024-04-17T07:57:22.989416+0000","last_became_active":"2024-04-17T07:55:25.504883+0000","last_became_peered":"2024-04-17T07:55:25.504883+0000","last_unstale":"2024-04-17T07:57:22.989416+0000","last_undegraded":"2024-04-17T07:57:22.989416+0000","last_fullsized":"2024-04-17T07:57:22.989416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:45.715121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990564+0000","last_change":"2024-04-17T07:56:22.833566+0000","last_active":"2024-04-17T07:57:22.990564+0000","last_peered":"2024-04-17T07:57:22.990564+0000","last_clean":"2024-04-17T07:57:22.990564+0000","last_became_active":"2024-04-17T07:56:22.833230+0000","last_became_peered":"2024-04-17T07:56:22.833230+0000","last_unstale":"2024-04-17T07:57:22.990564+0000","last_undegraded":"2024-04-17T07:57:22.990564+0000","last_fullsized":"2024-04-17T07:57:22.990564+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:42.333630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.985794+0000","last_change":"2024-04-17T07:55:30.606171+0000","last_active":"2024-04-17T07:57:22.985794+0000","last_peered":"2024-04-17T07:57:22.985794+0000","last_clean":"2024-04-17T07:57:22.985794+0000","last_became_active":"2024-04-17T07:55:30.605757+0000","last_became_peered":"2024-04-17T07:55:30.605757+0000","last_unstale":"2024-04-17T07:57:22.985794+0000","last_undegraded":"2024-04-17T07:57:22.985794+0000","last_fullsized":"2024-04-17T07:57:22.985794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:16.186650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981771+0000","last_change":"2024-04-17T07:55:26.353920+0000","last_active":"2024-04-17T07:57:22.981771+0000","last_peered":"2024-04-17T07:57:22.981771+0000","last_clean":"2024-04-17T07:57:22.981771+0000","last_became_active":"2024-04-17T07:55:26.352026+0000","last_became_peered":"2024-04-17T07:55:26.352026+0000","last_unstale":"2024-04-17T07:57:22.981771+0000","last_undegraded":"2024-04-17T07:57:22.981771+0000","last_fullsized":"2024-04-17T07:57:22.981771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:13:56.570420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":204,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:58:40.167483+0000","last_change":"2024-04-17T07:55:25.536760+0000","last_active":"2024-04-17T07:58:40.167483+0000","last_peered":"2024-04-17T07:58:40.167483+0000","last_clean":"2024-04-17T07:58:40.167483+0000","last_became_active":"2024-04-17T07:55:25.534892+0000","last_became_peered":"2024-04-17T07:55:25.534892+0000","last_unstale":"2024-04-17T07:58:40.167483+0000","last_undegraded":"2024-04-17T07:58:40.167483+0000","last_fullsized":"2024-04-17T07:58:40.167483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:20:30.708363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989218+0000","last_change":"2024-04-17T07:56:22.842324+0000","last_active":"2024-04-17T07:57:22.989218+0000","last_peered":"2024-04-17T07:57:22.989218+0000","last_clean":"2024-04-17T07:57:22.989218+0000","last_became_active":"2024-04-17T07:56:22.834023+0000","last_became_peered":"2024-04-17T07:56:22.834023+0000","last_unstale":"2024-04-17T07:57:22.989218+0000","last_undegraded":"2024-04-17T07:57:22.989218+0000","last_fullsized":"2024-04-17T07:57:22.989218+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:51:50.167121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982675+0000","last_change":"2024-04-17T07:55:26.375968+0000","last_active":"2024-04-17T07:57:22.982675+0000","last_peered":"2024-04-17T07:57:22.982675+0000","last_clean":"2024-04-17T07:57:22.982675+0000","last_became_active":"2024-04-17T07:55:26.373437+0000","last_became_peered":"2024-04-17T07:55:26.373437+0000","last_unstale":"2024-04-17T07:57:22.982675+0000","last_undegraded":"2024-04-17T07:57:22.982675+0000","last_fullsized":"2024-04-17T07:57:22.982675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:58:17.299639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987305+0000","last_change":"2024-04-17T07:55:25.523008+0000","last_active":"2024-04-17T07:57:22.987305+0000","last_peered":"2024-04-17T07:57:22.987305+0000","last_clean":"2024-04-17T07:57:22.987305+0000","last_became_active":"2024-04-17T07:55:25.522674+0000","last_became_peered":"2024-04-17T07:55:25.522674+0000","last_unstale":"2024-04-17T07:57:22.987305+0000","last_undegraded":"2024-04-17T07:57:22.987305+0000","last_fullsized":"2024-04-17T07:57:22.987305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:00.910380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989248+0000","last_change":"2024-04-17T07:54:04.894391+0000","last_active":"2024-04-17T07:57:22.989248+0000","last_peered":"2024-04-17T07:57:22.989248+0000","last_clean":"2024-04-17T07:57:22.989248+0000","last_became_active":"2024-04-17T07:54:04.893845+0000","last_became_peered":"2024-04-17T07:54:04.893845+0000","last_unstale":"2024-04-17T07:57:22.989248+0000","last_undegraded":"2024-04-17T07:57:22.989248+0000","last_fullsized":"2024-04-17T07:57:22.989248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:50:07.926945+0000","last_clean_scrub_stamp":"2024-04-17T07:50:07.926945+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:25.525015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982074+0000","last_change":"2024-04-17T07:56:22.825141+0000","last_active":"2024-04-17T07:57:22.982074+0000","last_peered":"2024-04-17T07:57:22.982074+0000","last_clean":"2024-04-17T07:57:22.982074+0000","last_became_active":"2024-04-17T07:56:22.824647+0000","last_became_peered":"2024-04-17T07:56:22.824647+0000","last_unstale":"2024-04-17T07:57:22.982074+0000","last_undegraded":"2024-04-17T07:57:22.982074+0000","last_fullsized":"2024-04-17T07:57:22.982074+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:15.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982417+0000","last_change":"2024-04-17T07:55:26.354120+0000","last_active":"2024-04-17T07:57:22.982417+0000","last_peered":"2024-04-17T07:57:22.982417+0000","last_clean":"2024-04-17T07:57:22.982417+0000","last_became_active":"2024-04-17T07:55:26.353671+0000","last_became_peered":"2024-04-17T07:55:26.353671+0000","last_unstale":"2024-04-17T07:57:22.982417+0000","last_undegraded":"2024-04-17T07:57:22.982417+0000","last_fullsized":"2024-04-17T07:57:22.982417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:50:25.420136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989408+0000","last_change":"2024-04-17T07:56:22.828216+0000","last_active":"2024-04-17T07:57:22.989408+0000","last_peered":"2024-04-17T07:57:22.989408+0000","last_clean":"2024-04-17T07:57:22.989408+0000","last_became_active":"2024-04-17T07:56:22.820969+0000","last_became_peered":"2024-04-17T07:56:22.820969+0000","last_unstale":"2024-04-17T07:57:22.989408+0000","last_undegraded":"2024-04-17T07:57:22.989408+0000","last_fullsized":"2024-04-17T07:57:22.989408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:35:08.046732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986358+0000","last_change":"2024-04-17T07:55:30.601882+0000","last_active":"2024-04-17T07:57:22.986358+0000","last_peered":"2024-04-17T07:57:22.986358+0000","last_clean":"2024-04-17T07:57:22.986358+0000","last_became_active":"2024-04-17T07:55:30.601387+0000","last_became_peered":"2024-04-17T07:55:30.601387+0000","last_unstale":"2024-04-17T07:57:22.986358+0000","last_undegraded":"2024-04-17T07:57:22.986358+0000","last_fullsized":"2024-04-17T07:57:22.986358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:25.860668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.991021+0000","last_change":"2024-04-17T07:55:25.528783+0000","last_active":"2024-04-17T07:57:22.991021+0000","last_peered":"2024-04-17T07:57:22.991021+0000","last_clean":"2024-04-17T07:57:22.991021+0000","last_became_active":"2024-04-17T07:55:25.528446+0000","last_became_peered":"2024-04-17T07:55:25.528446+0000","last_unstale":"2024-04-17T07:57:22.991021+0000","last_undegraded":"2024-04-17T07:57:22.991021+0000","last_fullsized":"2024-04-17T07:57:22.991021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:12.847570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983466+0000","last_change":"2024-04-17T07:55:26.368063+0000","last_active":"2024-04-17T07:57:22.983466+0000","last_peered":"2024-04-17T07:57:22.983466+0000","last_clean":"2024-04-17T07:57:22.983466+0000","last_became_active":"2024-04-17T07:55:26.367673+0000","last_became_peered":"2024-04-17T07:55:26.367673+0000","last_unstale":"2024-04-17T07:57:22.983466+0000","last_undegraded":"2024-04-17T07:57:22.983466+0000","last_fullsized":"2024-04-17T07:57:22.983466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:42.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981116+0000","last_change":"2024-04-17T07:55:30.598643+0000","last_active":"2024-04-17T07:57:22.981116+0000","last_peered":"2024-04-17T07:57:22.981116+0000","last_clean":"2024-04-17T07:57:22.981116+0000","last_became_active":"2024-04-17T07:55:30.596819+0000","last_became_peered":"2024-04-17T07:55:30.596819+0000","last_unstale":"2024-04-17T07:57:22.981116+0000","last_undegraded":"2024-04-17T07:57:22.981116+0000","last_fullsized":"2024-04-17T07:57:22.981116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:22.468302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'12","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:35.129665+0000","last_change":"2024-04-17T07:55:25.531979+0000","last_active":"2024-04-17T07:57:35.129665+0000","last_peered":"2024-04-17T07:57:35.129665+0000","last_clean":"2024-04-17T07:57:35.129665+0000","last_became_active":"2024-04-17T07:55:25.529191+0000","last_became_peered":"2024-04-17T07:55:25.529191+0000","last_unstale":"2024-04-17T07:57:35.129665+0000","last_undegraded":"2024-04-17T07:57:35.129665+0000","last_fullsized":"2024-04-17T07:57:35.129665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:54.011909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984249+0000","last_change":"2024-04-17T07:55:26.375036+0000","last_active":"2024-04-17T07:57:22.984249+0000","last_peered":"2024-04-17T07:57:22.984249+0000","last_clean":"2024-04-17T07:57:22.984249+0000","last_became_active":"2024-04-17T07:55:26.374668+0000","last_became_peered":"2024-04-17T07:55:26.374668+0000","last_unstale":"2024-04-17T07:57:22.984249+0000","last_undegraded":"2024-04-17T07:57:22.984249+0000","last_fullsized":"2024-04-17T07:57:22.984249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:48:40.397335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989826+0000","last_change":"2024-04-17T07:55:30.589030+0000","last_active":"2024-04-17T07:57:22.989826+0000","last_peered":"2024-04-17T07:57:22.989826+0000","last_clean":"2024-04-17T07:57:22.989826+0000","last_became_active":"2024-04-17T07:55:30.588448+0000","last_became_peered":"2024-04-17T07:55:30.588448+0000","last_unstale":"2024-04-17T07:57:22.989826+0000","last_undegraded":"2024-04-17T07:57:22.989826+0000","last_fullsized":"2024-04-17T07:57:22.989826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:22:06.741752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987958+0000","last_change":"2024-04-17T07:55:25.512216+0000","last_active":"2024-04-17T07:57:22.987958+0000","last_peered":"2024-04-17T07:57:22.987958+0000","last_clean":"2024-04-17T07:57:22.987958+0000","last_became_active":"2024-04-17T07:55:25.512004+0000","last_became_peered":"2024-04-17T07:55:25.512004+0000","last_unstale":"2024-04-17T07:57:22.987958+0000","last_undegraded":"2024-04-17T07:57:22.987958+0000","last_fullsized":"2024-04-17T07:57:22.987958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:51:25.524861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.301351+0000","last_change":"2024-04-17T07:55:26.368040+0000","last_active":"2024-04-17T07:57:23.301351+0000","last_peered":"2024-04-17T07:57:23.301351+0000","last_clean":"2024-04-17T07:57:23.301351+0000","last_became_active":"2024-04-17T07:55:26.367564+0000","last_became_peered":"2024-04-17T07:55:26.367564+0000","last_unstale":"2024-04-17T07:57:23.301351+0000","last_undegraded":"2024-04-17T07:57:23.301351+0000","last_fullsized":"2024-04-17T07:57:23.301351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:51.653353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987643+0000","last_change":"2024-04-17T07:55:30.592439+0000","last_active":"2024-04-17T07:57:22.987643+0000","last_peered":"2024-04-17T07:57:22.987643+0000","last_clean":"2024-04-17T07:57:22.987643+0000","last_became_active":"2024-04-17T07:55:30.591926+0000","last_became_peered":"2024-04-17T07:55:30.591926+0000","last_unstale":"2024-04-17T07:57:22.987643+0000","last_undegraded":"2024-04-17T07:57:22.987643+0000","last_fullsized":"2024-04-17T07:57:22.987643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:15:52.428856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297743+0000","last_change":"2024-04-17T07:55:25.535465+0000","last_active":"2024-04-17T07:57:23.297743+0000","last_peered":"2024-04-17T07:57:23.297743+0000","last_clean":"2024-04-17T07:57:23.297743+0000","last_became_active":"2024-04-17T07:55:25.535124+0000","last_became_peered":"2024-04-17T07:55:25.535124+0000","last_unstale":"2024-04-17T07:57:23.297743+0000","last_undegraded":"2024-04-17T07:57:23.297743+0000","last_fullsized":"2024-04-17T07:57:23.297743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:45:12.731320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989106+0000","last_change":"2024-04-17T07:55:26.344772+0000","last_active":"2024-04-17T07:57:22.989106+0000","last_peered":"2024-04-17T07:57:22.989106+0000","last_clean":"2024-04-17T07:57:22.989106+0000","last_became_active":"2024-04-17T07:55:26.344328+0000","last_became_peered":"2024-04-17T07:55:26.344328+0000","last_unstale":"2024-04-17T07:57:22.989106+0000","last_undegraded":"2024-04-17T07:57:22.989106+0000","last_fullsized":"2024-04-17T07:57:22.989106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:49.111861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299584+0000","last_change":"2024-04-17T07:55:30.605511+0000","last_active":"2024-04-17T07:57:23.299584+0000","last_peered":"2024-04-17T07:57:23.299584+0000","last_clean":"2024-04-17T07:57:23.299584+0000","last_became_active":"2024-04-17T07:55:30.604988+0000","last_became_peered":"2024-04-17T07:55:30.604988+0000","last_unstale":"2024-04-17T07:57:23.299584+0000","last_undegraded":"2024-04-17T07:57:23.299584+0000","last_fullsized":"2024-04-17T07:57:23.299584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:46.650370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981676+0000","last_change":"2024-04-17T07:55:25.532779+0000","last_active":"2024-04-17T07:57:22.981676+0000","last_peered":"2024-04-17T07:57:22.981676+0000","last_clean":"2024-04-17T07:57:22.981676+0000","last_became_active":"2024-04-17T07:55:25.527994+0000","last_became_peered":"2024-04-17T07:55:25.527994+0000","last_unstale":"2024-04-17T07:57:22.981676+0000","last_undegraded":"2024-04-17T07:57:22.981676+0000","last_fullsized":"2024-04-17T07:57:22.981676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:08:54.411665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986346+0000","last_change":"2024-04-17T07:55:26.359734+0000","last_active":"2024-04-17T07:57:22.986346+0000","last_peered":"2024-04-17T07:57:22.986346+0000","last_clean":"2024-04-17T07:57:22.986346+0000","last_became_active":"2024-04-17T07:55:26.359303+0000","last_became_peered":"2024-04-17T07:55:26.359303+0000","last_unstale":"2024-04-17T07:57:22.986346+0000","last_undegraded":"2024-04-17T07:57:22.986346+0000","last_fullsized":"2024-04-17T07:57:22.986346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:14.219658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987410+0000","last_change":"2024-04-17T07:55:30.604123+0000","last_active":"2024-04-17T07:57:22.987410+0000","last_peered":"2024-04-17T07:57:22.987410+0000","last_clean":"2024-04-17T07:57:22.987410+0000","last_became_active":"2024-04-17T07:55:30.603649+0000","last_became_peered":"2024-04-17T07:55:30.603649+0000","last_unstale":"2024-04-17T07:57:22.987410+0000","last_undegraded":"2024-04-17T07:57:22.987410+0000","last_fullsized":"2024-04-17T07:57:22.987410+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:42.243723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990346+0000","last_change":"2024-04-17T07:55:25.537324+0000","last_active":"2024-04-17T07:57:22.990346+0000","last_peered":"2024-04-17T07:57:22.990346+0000","last_clean":"2024-04-17T07:57:22.990346+0000","last_became_active":"2024-04-17T07:55:25.536347+0000","last_became_peered":"2024-04-17T07:55:25.536347+0000","last_unstale":"2024-04-17T07:57:22.990346+0000","last_undegraded":"2024-04-17T07:57:22.990346+0000","last_fullsized":"2024-04-17T07:57:22.990346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:58:31.810358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983638+0000","last_change":"2024-04-17T07:55:26.375448+0000","last_active":"2024-04-17T07:57:22.983638+0000","last_peered":"2024-04-17T07:57:22.983638+0000","last_clean":"2024-04-17T07:57:22.983638+0000","last_became_active":"2024-04-17T07:55:26.373220+0000","last_became_peered":"2024-04-17T07:55:26.373220+0000","last_unstale":"2024-04-17T07:57:22.983638+0000","last_undegraded":"2024-04-17T07:57:22.983638+0000","last_fullsized":"2024-04-17T07:57:22.983638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:55.216575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.987732+0000","last_change":"2024-04-17T07:55:30.600783+0000","last_active":"2024-04-17T07:57:22.987732+0000","last_peered":"2024-04-17T07:57:22.987732+0000","last_clean":"2024-04-17T07:57:22.987732+0000","last_became_active":"2024-04-17T07:55:30.600276+0000","last_became_peered":"2024-04-17T07:55:30.600276+0000","last_unstale":"2024-04-17T07:57:22.987732+0000","last_undegraded":"2024-04-17T07:57:22.987732+0000","last_fullsized":"2024-04-17T07:57:22.987732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:36.705881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989966+0000","last_change":"2024-04-17T07:56:22.825593+0000","last_active":"2024-04-17T07:57:22.989966+0000","last_peered":"2024-04-17T07:57:22.989966+0000","last_clean":"2024-04-17T07:57:22.989966+0000","last_became_active":"2024-04-17T07:56:22.819052+0000","last_became_peered":"2024-04-17T07:56:22.819052+0000","last_unstale":"2024-04-17T07:57:22.989966+0000","last_undegraded":"2024-04-17T07:57:22.989966+0000","last_fullsized":"2024-04-17T07:57:22.989966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:47:51.300879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989548+0000","last_change":"2024-04-17T07:55:26.356553+0000","last_active":"2024-04-17T07:57:22.989548+0000","last_peered":"2024-04-17T07:57:22.989548+0000","last_clean":"2024-04-17T07:57:22.989548+0000","last_became_active":"2024-04-17T07:55:26.356126+0000","last_became_peered":"2024-04-17T07:55:26.356126+0000","last_unstale":"2024-04-17T07:57:22.989548+0000","last_undegraded":"2024-04-17T07:57:22.989548+0000","last_fullsized":"2024-04-17T07:57:22.989548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:07.753059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.297780+0000","last_change":"2024-04-17T07:55:30.602881+0000","last_active":"2024-04-17T07:57:23.297780+0000","last_peered":"2024-04-17T07:57:23.297780+0000","last_clean":"2024-04-17T07:57:23.297780+0000","last_became_active":"2024-04-17T07:55:30.602423+0000","last_became_peered":"2024-04-17T07:55:30.602423+0000","last_unstale":"2024-04-17T07:57:23.297780+0000","last_undegraded":"2024-04-17T07:57:23.297780+0000","last_fullsized":"2024-04-17T07:57:23.297780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:38:20.223824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990064+0000","last_change":"2024-04-17T07:55:25.536887+0000","last_active":"2024-04-17T07:57:22.990064+0000","last_peered":"2024-04-17T07:57:22.990064+0000","last_clean":"2024-04-17T07:57:22.990064+0000","last_became_active":"2024-04-17T07:55:25.536134+0000","last_became_peered":"2024-04-17T07:55:25.536134+0000","last_unstale":"2024-04-17T07:57:22.990064+0000","last_undegraded":"2024-04-17T07:57:22.990064+0000","last_fullsized":"2024-04-17T07:57:22.990064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:42.791730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990382+0000","last_change":"2024-04-17T07:55:26.346081+0000","last_active":"2024-04-17T07:57:22.990382+0000","last_peered":"2024-04-17T07:57:22.990382+0000","last_clean":"2024-04-17T07:57:22.990382+0000","last_became_active":"2024-04-17T07:55:26.345601+0000","last_became_peered":"2024-04-17T07:55:26.345601+0000","last_unstale":"2024-04-17T07:57:22.990382+0000","last_undegraded":"2024-04-17T07:57:22.990382+0000","last_fullsized":"2024-04-17T07:57:22.990382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:38:44.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.299886+0000","last_change":"2024-04-17T07:55:30.591027+0000","last_active":"2024-04-17T07:57:23.299886+0000","last_peered":"2024-04-17T07:57:23.299886+0000","last_clean":"2024-04-17T07:57:23.299886+0000","last_became_active":"2024-04-17T07:55:30.590342+0000","last_became_peered":"2024-04-17T07:55:30.590342+0000","last_unstale":"2024-04-17T07:57:23.299886+0000","last_undegraded":"2024-04-17T07:57:23.299886+0000","last_fullsized":"2024-04-17T07:57:23.299886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:42.771629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.982144+0000","last_change":"2024-04-17T07:55:25.536028+0000","last_active":"2024-04-17T07:57:22.982144+0000","last_peered":"2024-04-17T07:57:22.982144+0000","last_clean":"2024-04-17T07:57:22.982144+0000","last_became_active":"2024-04-17T07:55:25.535730+0000","last_became_peered":"2024-04-17T07:55:25.535730+0000","last_unstale":"2024-04-17T07:57:22.982144+0000","last_undegraded":"2024-04-17T07:57:22.982144+0000","last_fullsized":"2024-04-17T07:57:22.982144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:38.714911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989834+0000","last_change":"2024-04-17T07:55:26.373301+0000","last_active":"2024-04-17T07:57:22.989834+0000","last_peered":"2024-04-17T07:57:22.989834+0000","last_clean":"2024-04-17T07:57:22.989834+0000","last_became_active":"2024-04-17T07:55:26.372965+0000","last_became_peered":"2024-04-17T07:55:26.372965+0000","last_unstale":"2024-04-17T07:57:22.989834+0000","last_undegraded":"2024-04-17T07:57:22.989834+0000","last_fullsized":"2024-04-17T07:57:22.989834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:22.436945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981278+0000","last_change":"2024-04-17T07:55:30.590587+0000","last_active":"2024-04-17T07:57:22.981278+0000","last_peered":"2024-04-17T07:57:22.981278+0000","last_clean":"2024-04-17T07:57:22.981278+0000","last_became_active":"2024-04-17T07:55:30.590127+0000","last_became_peered":"2024-04-17T07:55:30.590127+0000","last_unstale":"2024-04-17T07:57:22.981278+0000","last_undegraded":"2024-04-17T07:57:22.981278+0000","last_fullsized":"2024-04-17T07:57:22.981278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:29.556899+0000","last_clean_scrub_stamp":"2024-04-17T07:55:29.556899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.010494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984913+0000","last_change":"2024-04-17T07:56:23.591665+0000","last_active":"2024-04-17T07:57:22.984913+0000","last_peered":"2024-04-17T07:57:22.984913+0000","last_clean":"2024-04-17T07:57:22.984913+0000","last_became_active":"2024-04-17T07:56:23.584168+0000","last_became_peered":"2024-04-17T07:56:23.584168+0000","last_unstale":"2024-04-17T07:57:22.984913+0000","last_undegraded":"2024-04-17T07:57:22.984913+0000","last_fullsized":"2024-04-17T07:57:22.984913+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:24.479184+0000","last_clean_scrub_stamp":"2024-04-17T07:55:24.479184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:16.727325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.989950+0000","last_change":"2024-04-17T07:55:26.348067+0000","last_active":"2024-04-17T07:57:22.989950+0000","last_peered":"2024-04-17T07:57:22.989950+0000","last_clean":"2024-04-17T07:57:22.989950+0000","last_became_active":"2024-04-17T07:55:26.347598+0000","last_became_peered":"2024-04-17T07:55:26.347598+0000","last_unstale":"2024-04-17T07:57:22.989950+0000","last_undegraded":"2024-04-17T07:57:22.989950+0000","last_fullsized":"2024-04-17T07:57:22.989950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:22.856985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986499+0000","last_change":"2024-04-17T07:55:26.351277+0000","last_active":"2024-04-17T07:57:22.986499+0000","last_peered":"2024-04-17T07:57:22.986499+0000","last_clean":"2024-04-17T07:57:22.986499+0000","last_became_active":"2024-04-17T07:55:26.350599+0000","last_became_peered":"2024-04-17T07:55:26.350599+0000","last_unstale":"2024-04-17T07:57:22.986499+0000","last_undegraded":"2024-04-17T07:57:22.986499+0000","last_fullsized":"2024-04-17T07:57:22.986499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:56.649103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990103+0000","last_change":"2024-04-17T07:55:26.352642+0000","last_active":"2024-04-17T07:57:22.990103+0000","last_peered":"2024-04-17T07:57:22.990103+0000","last_clean":"2024-04-17T07:57:22.990103+0000","last_became_active":"2024-04-17T07:55:26.352290+0000","last_became_peered":"2024-04-17T07:55:26.352290+0000","last_unstale":"2024-04-17T07:57:22.990103+0000","last_undegraded":"2024-04-17T07:57:22.990103+0000","last_fullsized":"2024-04-17T07:57:22.990103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:29.436515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990216+0000","last_change":"2024-04-17T07:55:26.370958+0000","last_active":"2024-04-17T07:57:22.990216+0000","last_peered":"2024-04-17T07:57:22.990216+0000","last_clean":"2024-04-17T07:57:22.990216+0000","last_became_active":"2024-04-17T07:55:26.370642+0000","last_became_peered":"2024-04-17T07:55:26.370642+0000","last_unstale":"2024-04-17T07:57:22.990216+0000","last_undegraded":"2024-04-17T07:57:22.990216+0000","last_fullsized":"2024-04-17T07:57:22.990216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:27.766660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.300391+0000","last_change":"2024-04-17T07:55:26.365113+0000","last_active":"2024-04-17T07:57:23.300391+0000","last_peered":"2024-04-17T07:57:23.300391+0000","last_clean":"2024-04-17T07:57:23.300391+0000","last_became_active":"2024-04-17T07:55:26.364770+0000","last_became_peered":"2024-04-17T07:55:26.364770+0000","last_unstale":"2024-04-17T07:57:23.300391+0000","last_undegraded":"2024-04-17T07:57:23.300391+0000","last_fullsized":"2024-04-17T07:57:23.300391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:44.060092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.983162+0000","last_change":"2024-04-17T07:55:26.361235+0000","last_active":"2024-04-17T07:57:22.983162+0000","last_peered":"2024-04-17T07:57:22.983162+0000","last_clean":"2024-04-17T07:57:22.983162+0000","last_became_active":"2024-04-17T07:55:26.357506+0000","last_became_peered":"2024-04-17T07:55:26.357506+0000","last_unstale":"2024-04-17T07:57:22.983162+0000","last_undegraded":"2024-04-17T07:57:22.983162+0000","last_fullsized":"2024-04-17T07:57:22.983162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:51.300146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986726+0000","last_change":"2024-04-17T07:55:26.350972+0000","last_active":"2024-04-17T07:57:22.986726+0000","last_peered":"2024-04-17T07:57:22.986726+0000","last_clean":"2024-04-17T07:57:22.986726+0000","last_became_active":"2024-04-17T07:55:26.350571+0000","last_became_peered":"2024-04-17T07:55:26.350571+0000","last_unstale":"2024-04-17T07:57:22.986726+0000","last_undegraded":"2024-04-17T07:57:22.986726+0000","last_fullsized":"2024-04-17T07:57:22.986726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:19.895639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.990569+0000","last_change":"2024-04-17T07:55:26.351031+0000","last_active":"2024-04-17T07:57:22.990569+0000","last_peered":"2024-04-17T07:57:22.990569+0000","last_clean":"2024-04-17T07:57:22.990569+0000","last_became_active":"2024-04-17T07:55:26.350693+0000","last_became_peered":"2024-04-17T07:55:26.350693+0000","last_unstale":"2024-04-17T07:57:22.990569+0000","last_undegraded":"2024-04-17T07:57:22.990569+0000","last_fullsized":"2024-04-17T07:57:22.990569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:51.486776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984737+0000","last_change":"2024-04-17T07:55:26.373663+0000","last_active":"2024-04-17T07:57:22.984737+0000","last_peered":"2024-04-17T07:57:22.984737+0000","last_clean":"2024-04-17T07:57:22.984737+0000","last_became_active":"2024-04-17T07:55:26.373252+0000","last_became_peered":"2024-04-17T07:55:26.373252+0000","last_unstale":"2024-04-17T07:57:22.984737+0000","last_undegraded":"2024-04-17T07:57:22.984737+0000","last_fullsized":"2024-04-17T07:57:22.984737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:23:54.986349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981744+0000","last_change":"2024-04-17T07:55:26.373194+0000","last_active":"2024-04-17T07:57:22.981744+0000","last_peered":"2024-04-17T07:57:22.981744+0000","last_clean":"2024-04-17T07:57:22.981744+0000","last_became_active":"2024-04-17T07:55:26.372837+0000","last_became_peered":"2024-04-17T07:55:26.372837+0000","last_unstale":"2024-04-17T07:57:22.981744+0000","last_undegraded":"2024-04-17T07:57:22.981744+0000","last_fullsized":"2024-04-17T07:57:22.981744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:41.077779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.981978+0000","last_change":"2024-04-17T07:55:26.359978+0000","last_active":"2024-04-17T07:57:22.981978+0000","last_peered":"2024-04-17T07:57:22.981978+0000","last_clean":"2024-04-17T07:57:22.981978+0000","last_became_active":"2024-04-17T07:55:26.359384+0000","last_became_peered":"2024-04-17T07:55:26.359384+0000","last_unstale":"2024-04-17T07:57:22.981978+0000","last_undegraded":"2024-04-17T07:57:22.981978+0000","last_fullsized":"2024-04-17T07:57:22.981978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:25:37.872008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:23.298052+0000","last_change":"2024-04-17T07:55:26.368777+0000","last_active":"2024-04-17T07:57:23.298052+0000","last_peered":"2024-04-17T07:57:23.298052+0000","last_clean":"2024-04-17T07:57:23.298052+0000","last_became_active":"2024-04-17T07:55:26.368367+0000","last_became_peered":"2024-04-17T07:55:26.368367+0000","last_unstale":"2024-04-17T07:57:23.298052+0000","last_undegraded":"2024-04-17T07:57:23.298052+0000","last_fullsized":"2024-04-17T07:57:23.298052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:52.132244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.984677+0000","last_change":"2024-04-17T07:55:26.360669+0000","last_active":"2024-04-17T07:57:22.984677+0000","last_peered":"2024-04-17T07:57:22.984677+0000","last_clean":"2024-04-17T07:57:22.984677+0000","last_became_active":"2024-04-17T07:55:26.360314+0000","last_became_peered":"2024-04-17T07:55:26.360314+0000","last_unstale":"2024-04-17T07:57:22.984677+0000","last_undegraded":"2024-04-17T07:57:22.984677+0000","last_fullsized":"2024-04-17T07:57:22.984677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:07.223891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:57:22.986573+0000","last_change":"2024-04-17T07:55:26.351095+0000","last_active":"2024-04-17T07:57:22.986573+0000","last_peered":"2024-04-17T07:57:22.986573+0000","last_clean":"2024-04-17T07:57:22.986573+0000","last_became_active":"2024-04-17T07:55:26.350689+0000","last_became_peered":"2024-04-17T07:55:26.350689+0000","last_unstale":"2024-04-17T07:57:22.986573+0000","last_undegraded":"2024-04-17T07:57:22.986573+0000","last_fullsized":"2024-04-17T07:57:22.986573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:55:25.487619+0000","last_clean_scrub_stamp":"2024-04-17T07:55:25.487619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:20:11.599304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73231,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1035,"num_read_kb":598,"num_write":179,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1120,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":219693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383735,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":520709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1175946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":523921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070535,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":523841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332172,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1115207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495698,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":515001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":578095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":585350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":515195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":520183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1671168,"data_stored":1107287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:58:46.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:58:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:46 smithi046 ceph-mon[32038]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:46 smithi046 ceph-mon[32038]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:46.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:46 smithi046 ceph-mon[32038]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 -- 172.21.15.46:0/3993517503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0074940 msgr2=0x7f90c0074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 --2- 172.21.15.46:0/3993517503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0074940 0x7f90c0074da0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009860 tx=0x7f90a802bc00 comp rx=0 tx=0).stop 2024-04-17T07:58:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 -- 172.21.15.46:0/3993517503 shutdown_connections 2024-04-17T07:58:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 --2- 172.21.15.46:0/3993517503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 0x7f90c010c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 --2- 172.21.15.46:0/3993517503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0074940 0x7f90c0074da0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 --2- 172.21.15.46:0/3993517503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90c0075f20 0x7f90c0074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 -- 172.21.15.46:0/3993517503 >> 172.21.15.46:0/3993517503 conn(0x7f90c00fd590 msgr2=0x7f90c00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 -- 172.21.15.46:0/3993517503 shutdown_connections 2024-04-17T07:58:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.375+0000 7f90c6200640 1 -- 172.21.15.46:0/3993517503 wait complete. 2024-04-17T07:58:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.376+0000 7f90c6200640 1 Processor -- start 2024-04-17T07:58:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90c6200640 1 -- start start 2024-04-17T07:58:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90c6200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90c0074940 0x7f90c019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90c6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0075f20 0x7f90c019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90c6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 0x7f90c01a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90c6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010fe50 con 0x7f90c0075f20 2024-04-17T07:58:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90c6200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010fcd0 con 0x7f90c0074940 2024-04-17T07:58:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90c6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010ffd0 con 0x7f90c0109ed0 2024-04-17T07:58:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 0x7f90c01a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0075f20 0x7f90c019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90bf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90c0074940 0x7f90c019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 0x7f90c01a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:38532/0 (socket says 172.21.15.46:38532) 2024-04-17T07:58:46.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.377+0000 7f90bfe00640 1 -- 172.21.15.46:0/454707629 learned_addr learned my addr 172.21.15.46:0/454707629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.378+0000 7f90bfe00640 1 -- 172.21.15.46:0/454707629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90c0074940 msgr2=0x7f90c019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.378+0000 7f90bfe00640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90c0074940 0x7f90c019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.378+0000 7f90bfe00640 1 -- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0075f20 msgr2=0x7f90c019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.378+0000 7f90bfe00640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0075f20 0x7f90c019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.378+0000 7f90bfe00640 1 -- 172.21.15.46:0/454707629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c01a20a0 con 0x7f90c0109ed0 2024-04-17T07:58:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.378+0000 7f90bfe00640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 0x7f90c01a1a30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90b800e970 tx=0x7f90b800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.378+0000 7f90b7e00640 1 -- 172.21.15.46:0/454707629 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b800ccc0 con 0x7f90c0109ed0 2024-04-17T07:58:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.379+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c01a2390 con 0x7f90c0109ed0 2024-04-17T07:58:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.379+0000 7f90b7e00640 1 -- 172.21.15.46:0/454707629 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b800ce60 con 0x7f90c0109ed0 2024-04-17T07:58:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.379+0000 7f90b7e00640 1 -- 172.21.15.46:0/454707629 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8010450 con 0x7f90c0109ed0 2024-04-17T07:58:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.379+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01a2900 con 0x7f90c0109ed0 2024-04-17T07:58:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.380+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c0108c90 con 0x7f90c0109ed0 2024-04-17T07:58:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.382+0000 7f90b7e00640 1 -- 172.21.15.46:0/454707629 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f90b8005020 con 0x7f90c0109ed0 2024-04-17T07:58:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.384+0000 7f90b7e00640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90a40a3c30 0x7f90a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.384+0000 7f90b7e00640 1 -- 172.21.15.46:0/454707629 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f90b80d0b40 con 0x7f90c0109ed0 2024-04-17T07:58:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.384+0000 7f90bf400640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90a40a3c30 0x7f90a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.385+0000 7f90bf400640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90a40a3c30 0x7f90a40a60f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009a60 tx=0x7f90b0010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.388+0000 7f90b7e00640 1 -- 172.21.15.46:0/454707629 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f90b809d190 con 0x7f90c0109ed0 2024-04-17T07:58:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.596+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90c0062b60 con 0x7f90c0109ed0 2024-04-17T07:58:46.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.598+0000 7f90b7e00640 1 -- 172.21.15.46:0/454707629 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7f90b80bd0d0 con 0x7f90c0109ed0 2024-04-17T07:58:46.598 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:46.598 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90a40a3c30 msgr2=0x7f90a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90a40a3c30 0x7f90a40a60f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009a60 tx=0x7f90b0010040 comp rx=0 tx=0).stop 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 msgr2=0x7f90c01a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 0x7f90c01a1a30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90b800e970 tx=0x7f90b800ed70 comp rx=0 tx=0).stop 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 shutdown_connections 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90a40a3c30 0x7f90a40a60f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90c0109ed0 0x7f90c01a1a30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c0075f20 0x7f90c019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 --2- 172.21.15.46:0/454707629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90c0074940 0x7f90c019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.601+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 >> 172.21.15.46:0/454707629 conn(0x7f90c00fd590 msgr2=0x7f90c0105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.602+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 shutdown_connections 2024-04-17T07:58:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.602+0000 7f90c6200640 1 -- 172.21.15.46:0/454707629 wait complete. 2024-04-17T07:58:46.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:58:46.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:58:46.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:46 smithi053 ceph-mon[36437]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:46.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:46 smithi053 ceph-mon[36437]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:46.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:46 smithi053 ceph-mon[36437]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:46.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:46 smithi062 ceph-mon[36507]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:46 smithi062 ceph-mon[36507]: pgmap v425: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:46.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:46 smithi062 ceph-mon[36507]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:58:46.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 -- 172.21.15.46:0/1080273286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468102fc0 msgr2=0x7f7468103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 --2- 172.21.15.46:0/1080273286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468102fc0 0x7f7468103440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7450009900 tx=0x7f745002bc30 comp rx=0 tx=0).stop 2024-04-17T07:58:46.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 -- 172.21.15.46:0/1080273286 shutdown_connections 2024-04-17T07:58:46.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 --2- 172.21.15.46:0/1080273286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7468103980 0x7f746810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 --2- 172.21.15.46:0/1080273286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468102fc0 0x7f7468103440 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 --2- 172.21.15.46:0/1080273286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468101dd0 0x7f74681021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 -- 172.21.15.46:0/1080273286 >> 172.21.15.46:0/1080273286 conn(0x7f74680fd340 msgr2=0x7f74680ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:46.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 -- 172.21.15.46:0/1080273286 shutdown_connections 2024-04-17T07:58:46.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.851+0000 7f746ca00640 1 -- 172.21.15.46:0/1080273286 wait complete. 2024-04-17T07:58:46.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.852+0000 7f746ca00640 1 Processor -- start 2024-04-17T07:58:46.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.852+0000 7f746ca00640 1 -- start start 2024-04-17T07:58:46.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.852+0000 7f746ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7468101dd0 0x7f7468198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.852+0000 7f746ca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468102fc0 0x7f74681985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.852+0000 7f746ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468103980 0x7f746819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f746ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f746810daa0 con 0x7f7468103980 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f746ca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f746810d920 con 0x7f7468102fc0 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f746ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f746810dc20 con 0x7f7468101dd0 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7468101dd0 0x7f7468198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7468101dd0 0x7f7468198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:38552/0 (socket says 172.21.15.46:38552) 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467400640 1 -- 172.21.15.46:0/1606839116 learned_addr learned my addr 172.21.15.46:0/1606839116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467e00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468103980 0x7f746819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7466a00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468102fc0 0x7f74681985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467e00640 1 -- 172.21.15.46:0/1606839116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7468101dd0 msgr2=0x7f7468198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467e00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7468101dd0 0x7f7468198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467e00640 1 -- 172.21.15.46:0/1606839116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468102fc0 msgr2=0x7f74681985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467e00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468102fc0 0x7f74681985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:46.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7467e00640 1 -- 172.21.15.46:0/1606839116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746819fca0 con 0x7f7468103980 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.853+0000 7f7466a00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468102fc0 0x7f74681985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.854+0000 7f7467e00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468103980 0x7f746819f630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7460009fd0 tx=0x7f746000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.854+0000 7f745fe00640 1 -- 172.21.15.46:0/1606839116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746000ce20 con 0x7f7468103980 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.854+0000 7f745fe00640 1 -- 172.21.15.46:0/1606839116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7460005020 con 0x7f7468103980 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.854+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746819ff90 con 0x7f7468103980 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.854+0000 7f745fe00640 1 -- 172.21.15.46:0/1606839116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460010720 con 0x7f7468103980 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.854+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681a0450 con 0x7f7468103980 2024-04-17T07:58:46.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.855+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7468109090 con 0x7f7468103980 2024-04-17T07:58:46.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.857+0000 7f745fe00640 1 -- 172.21.15.46:0/1606839116 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7460005540 con 0x7f7468103980 2024-04-17T07:58:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.860+0000 7f745fe00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f744c0a3ba0 0x7f744c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.860+0000 7f745fe00640 1 -- 172.21.15.46:0/1606839116 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f74600d0e00 con 0x7f7468103980 2024-04-17T07:58:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.860+0000 7f7467400640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f744c0a3ba0 0x7f744c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.861+0000 7f7467400640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f744c0a3ba0 0x7f744c0a6060 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74580058c0 tx=0x7f7458005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:46.865+0000 7f745fe00640 1 -- 172.21.15.46:0/1606839116 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f746009d3f0 con 0x7f7468103980 2024-04-17T07:58:47.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.073+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7468062c00 con 0x7f7468103980 2024-04-17T07:58:47.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.074+0000 7f745fe00640 1 -- 172.21.15.46:0/1606839116 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f746009ccf0 con 0x7f7468103980 2024-04-17T07:58:47.073 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T07:58:47.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f744c0a3ba0 msgr2=0x7f744c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f744c0a3ba0 0x7f744c0a6060 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74580058c0 tx=0x7f7458005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:47.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468103980 msgr2=0x7f746819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468103980 0x7f746819f630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7460009fd0 tx=0x7f746000ef10 comp rx=0 tx=0).stop 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 shutdown_connections 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f744c0a3ba0 0x7f744c0a6060 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468103980 0x7f746819f630 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7468102fc0 0x7f74681985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 --2- 172.21.15.46:0/1606839116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7468101dd0 0x7f7468198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 >> 172.21.15.46:0/1606839116 conn(0x7f74680fd340 msgr2=0x7f7468106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 shutdown_connections 2024-04-17T07:58:47.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.077+0000 7f746ca00640 1 -- 172.21.15.46:0/1606839116 wait complete. 2024-04-17T07:58:47.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:58:47.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.331+0000 7f189da00640 1 -- 172.21.15.46:0/743350348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 msgr2=0x7f1898104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.331+0000 7f189da00640 1 --2- 172.21.15.46:0/743350348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f1898104420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1888009900 tx=0x7f188802bc50 comp rx=0 tx=0).stop 2024-04-17T07:58:47.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.331+0000 7f189da00640 1 -- 172.21.15.46:0/743350348 shutdown_connections 2024-04-17T07:58:47.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.331+0000 7f189da00640 1 --2- 172.21.15.46:0/743350348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1898105bb0 0x7f189810c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.331+0000 7f189da00640 1 --2- 172.21.15.46:0/743350348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18981051f0 0x7f1898105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.331+0000 7f189da00640 1 --2- 172.21.15.46:0/743350348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f1898104420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.331+0000 7f189da00640 1 -- 172.21.15.46:0/743350348 >> 172.21.15.46:0/743350348 conn(0x7f18980ff570 msgr2=0x7f18981019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:47.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.332+0000 7f189da00640 1 -- 172.21.15.46:0/743350348 shutdown_connections 2024-04-17T07:58:47.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.332+0000 7f189da00640 1 -- 172.21.15.46:0/743350348 wait complete. 2024-04-17T07:58:47.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 Processor -- start 2024-04-17T07:58:47.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 -- start start 2024-04-17T07:58:47.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f189819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18981051f0 0x7f189819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1898105bb0 0x7f18981a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f189810fca0 con 0x7f1898104000 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f189810fb20 con 0x7f18981051f0 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f189810fe20 con 0x7f1898105bb0 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f1897e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f189819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f1897e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f189819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60712/0 (socket says 172.21.15.46:60712) 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f1897e00640 1 -- 172.21.15.46:0/3111112715 learned_addr learned my addr 172.21.15.46:0/3111112715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f1897400640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18981051f0 0x7f189819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.333+0000 7f189d000640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1898105bb0 0x7f18981a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1897e00640 1 -- 172.21.15.46:0/3111112715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1898105bb0 msgr2=0x7f18981a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1897e00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1898105bb0 0x7f18981a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1897e00640 1 -- 172.21.15.46:0/3111112715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18981051f0 msgr2=0x7f189819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1897e00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18981051f0 0x7f189819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1897e00640 1 -- 172.21.15.46:0/3111112715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18981a1f30 con 0x7f1898104000 2024-04-17T07:58:47.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1897e00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f189819a310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f188802c260 tx=0x7f188802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:47.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1894c00640 1 -- 172.21.15.46:0/3111112715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888041070 con 0x7f1898104000 2024-04-17T07:58:47.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1894c00640 1 -- 172.21.15.46:0/3111112715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1888004450 con 0x7f1898104000 2024-04-17T07:58:47.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f1894c00640 1 -- 172.21.15.46:0/3111112715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888034440 con 0x7f1898104000 2024-04-17T07:58:47.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.334+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18981a21c0 con 0x7f1898104000 2024-04-17T07:58:47.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.335+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a26c0 con 0x7f1898104000 2024-04-17T07:58:47.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.336+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189810b320 con 0x7f1898104000 2024-04-17T07:58:47.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.338+0000 7f1894c00640 1 -- 172.21.15.46:0/3111112715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1888044070 con 0x7f1898104000 2024-04-17T07:58:47.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.339+0000 7f1894c00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f187c0a3bb0 0x7f187c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.339+0000 7f1894c00640 1 -- 172.21.15.46:0/3111112715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f18880f2270 con 0x7f1898104000 2024-04-17T07:58:47.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.340+0000 7f1897400640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f187c0a3bb0 0x7f187c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.340+0000 7f1897400640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f187c0a3bb0 0x7f187c0a6070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f189819b820 tx=0x7f1884005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:47.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.344+0000 7f1894c00640 1 -- 172.21.15.46:0/3111112715 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f18880be210 con 0x7f1898104000 2024-04-17T07:58:47.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.553+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1898062b60 con 0x7f1898104000 2024-04-17T07:58:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.554+0000 7f1894c00640 1 -- 172.21.15.46:0/3111112715 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f18880c2510 con 0x7f1898104000 2024-04-17T07:58:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f187c0a3bb0 msgr2=0x7f187c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f187c0a3bb0 0x7f187c0a6070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f189819b820 tx=0x7f1884005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 msgr2=0x7f189819a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f189819a310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f188802c260 tx=0x7f188802c670 comp rx=0 tx=0).stop 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 shutdown_connections 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f187c0a3bb0 0x7f187c0a6070 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1898105bb0 0x7f18981a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18981051f0 0x7f189819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 --2- 172.21.15.46:0/3111112715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1898104000 0x7f189819a310 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 >> 172.21.15.46:0/3111112715 conn(0x7f18980ff570 msgr2=0x7f18981084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 shutdown_connections 2024-04-17T07:58:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.557+0000 7f189da00640 1 -- 172.21.15.46:0/3111112715 wait complete. 2024-04-17T07:58:47.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:58:47.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:47 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/454707629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:47.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:47 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1606839116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:47.622 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:47 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/454707629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:47.622 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:47 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1606839116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:47.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:47 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/454707629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:47.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:47 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1606839116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 -- 172.21.15.46:0/842083194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94109c70 msgr2=0x7f9c9410c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/842083194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94109c70 0x7f9c9410c100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009ca0 tx=0x7f9c8c02c100 comp rx=0 tx=0).stop 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 -- 172.21.15.46:0/842083194 shutdown_connections 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/842083194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94109c70 0x7f9c9410c100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/842083194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 0x7f9c94074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/842083194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c94075fa0 0x7f9c940743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 -- 172.21.15.46:0/842083194 >> 172.21.15.46:0/842083194 conn(0x7f9c940fd320 msgr2=0x7f9c940ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 -- 172.21.15.46:0/842083194 shutdown_connections 2024-04-17T07:58:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.806+0000 7f9c9aa00640 1 -- 172.21.15.46:0/842083194 wait complete. 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 Processor -- start 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 -- start start 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 0x7f9c9419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94075fa0 0x7f9c9419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c94109c70 0x7f9c941a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9410fc30 con 0x7f9c94109c70 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9410fab0 con 0x7f9c940749c0 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c9aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9410fdb0 con 0x7f9c94075fa0 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.808+0000 7f9c99600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 0x7f9c9419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c9a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c94109c70 0x7f9c941a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c98c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94075fa0 0x7f9c9419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 0x7f9c9419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:50232/0 (socket says 172.21.15.46:50232) 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c98c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94075fa0 0x7f9c9419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:38598/0 (socket says 172.21.15.46:38598) 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 -- 172.21.15.46:0/1253616640 learned_addr learned my addr 172.21.15.46:0/1253616640 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 -- 172.21.15.46:0/1253616640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94075fa0 msgr2=0x7f9c9419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94075fa0 0x7f9c9419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 -- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c94109c70 msgr2=0x7f9c941a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c94109c70 0x7f9c941a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 -- 172.21.15.46:0/1253616640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941a1e40 con 0x7f9c940749c0 2024-04-17T07:58:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.809+0000 7f9c99600640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 0x7f9c9419a220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d950 tx=0x7f9c8800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.810+0000 7f9c92000640 1 -- 172.21.15.46:0/1253616640 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017030 con 0x7f9c940749c0 2024-04-17T07:58:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.810+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941a2130 con 0x7f9c940749c0 2024-04-17T07:58:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.810+0000 7f9c92000640 1 -- 172.21.15.46:0/1253616640 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c88004020 con 0x7f9c940749c0 2024-04-17T07:58:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.810+0000 7f9c92000640 1 -- 172.21.15.46:0/1253616640 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880047a0 con 0x7f9c940749c0 2024-04-17T07:58:47.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.810+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941a26d0 con 0x7f9c940749c0 2024-04-17T07:58:47.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.811+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c941089a0 con 0x7f9c940749c0 2024-04-17T07:58:47.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.813+0000 7f9c92000640 1 -- 172.21.15.46:0/1253616640 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9c880041d0 con 0x7f9c940749c0 2024-04-17T07:58:47.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.814+0000 7f9c92000640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c7c0a3c20 0x7f9c7c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:47.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.814+0000 7f9c92000640 1 -- 172.21.15.46:0/1253616640 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9c880cff40 con 0x7f9c940749c0 2024-04-17T07:58:47.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.815+0000 7f9c98c00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c7c0a3c20 0x7f9c7c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.815+0000 7f9c98c00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c7c0a3c20 0x7f9c7c0a60e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9c9419b730 tx=0x7f9c80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:47.820+0000 7f9c92000640 1 -- 172.21.15.46:0/1253616640 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9c88022020 con 0x7f9c940749c0 2024-04-17T07:58:48.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.029+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9c94062b60 con 0x7f9c940749c0 2024-04-17T07:58:48.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.030+0000 7f9c92000640 1 -- 172.21.15.46:0/1253616640 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f9c8809c5c0 con 0x7f9c940749c0 2024-04-17T07:58:48.029 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.032+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c7c0a3c20 msgr2=0x7f9c7c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.032+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c7c0a3c20 0x7f9c7c0a60e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9c9419b730 tx=0x7f9c80005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.032+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 msgr2=0x7f9c9419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.032+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 0x7f9c9419a220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d950 tx=0x7f9c8800de10 comp rx=0 tx=0).stop 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.032+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 shutdown_connections 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.033+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9c7c0a3c20 0x7f9c7c0a60e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.033+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c94109c70 0x7f9c941a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.033+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c94075fa0 0x7f9c9419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.033+0000 7f9c9aa00640 1 --2- 172.21.15.46:0/1253616640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c940749c0 0x7f9c9419a220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.033+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 >> 172.21.15.46:0/1253616640 conn(0x7f9c940fd320 msgr2=0x7f9c94105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:48.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.033+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 shutdown_connections 2024-04-17T07:58:48.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.033+0000 7f9c9aa00640 1 -- 172.21.15.46:0/1253616640 wait complete. 2024-04-17T07:58:48.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:58:48.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 -- 172.21.15.46:0/1777249318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980102850 msgr2=0x7f0980104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 --2- 172.21.15.46:0/1777249318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980102850 0x7f0980104c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0974009ca0 tx=0x7f097402c100 comp rx=0 tx=0).stop 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 -- 172.21.15.46:0/1777249318 shutdown_connections 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 --2- 172.21.15.46:0/1777249318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0980107ab0 0x7f0980109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 --2- 172.21.15.46:0/1777249318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980105180 0x7f0980107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 --2- 172.21.15.46:0/1777249318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980102850 0x7f0980104c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 -- 172.21.15.46:0/1777249318 >> 172.21.15.46:0/1777249318 conn(0x7f09800fc240 msgr2=0x7f09800fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 -- 172.21.15.46:0/1777249318 shutdown_connections 2024-04-17T07:58:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.325+0000 7f0986c00640 1 -- 172.21.15.46:0/1777249318 wait complete. 2024-04-17T07:58:48.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.326+0000 7f0986c00640 1 Processor -- start 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.326+0000 7f0986c00640 1 -- start start 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0986c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102850 0x7f0980195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0986c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980105180 0x7f0980196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0986c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0980107ab0 0x7f09801a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0986c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f098010da50 con 0x7f0980105180 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0986c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f098010d8d0 con 0x7f0980102850 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0986c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f098010dbd0 con 0x7f0980107ab0 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0985800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102850 0x7f0980195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0984e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980105180 0x7f0980196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0985800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102850 0x7f0980195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42636/0 (socket says 172.21.15.46:42636) 2024-04-17T07:58:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0985800640 1 -- 172.21.15.46:0/1557835917 learned_addr learned my addr 172.21.15.46:0/1557835917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.327+0000 7f0986200640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0980107ab0 0x7f09801a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0985800640 1 -- 172.21.15.46:0/1557835917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0980107ab0 msgr2=0x7f09801a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0985800640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0980107ab0 0x7f09801a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0985800640 1 -- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980105180 msgr2=0x7f0980196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0985800640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980105180 0x7f0980196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0985800640 1 -- 172.21.15.46:0/1557835917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09801a6370 con 0x7f0980102850 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0985800640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102850 0x7f0980195f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09740325f0 tx=0x7f0974035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f097e000640 1 -- 172.21.15.46:0/1557835917 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974039040 con 0x7f0980102850 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09801a6600 con 0x7f0980102850 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f097e000640 1 -- 172.21.15.46:0/1557835917 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0974044070 con 0x7f0980102850 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f097e000640 1 -- 172.21.15.46:0/1557835917 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974004460 con 0x7f0980102850 2024-04-17T07:58:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.328+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09801a6b10 con 0x7f0980102850 2024-04-17T07:58:48.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.330+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0980109600 con 0x7f0980102850 2024-04-17T07:58:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.332+0000 7f097e000640 1 -- 172.21.15.46:0/1557835917 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0974004600 con 0x7f0980102850 2024-04-17T07:58:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.333+0000 7f097e000640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09680a3bb0 0x7f09680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.334+0000 7f097e000640 1 -- 172.21.15.46:0/1557835917 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f09740f3590 con 0x7f0980102850 2024-04-17T07:58:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.334+0000 7f0984e00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09680a3bb0 0x7f09680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.334+0000 7f0984e00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09680a3bb0 0x7f09680a6070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0980197510 tx=0x7f096c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:48.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.336+0000 7f097e000640 1 -- 172.21.15.46:0/1557835917 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f097404e020 con 0x7f0980102850 2024-04-17T07:58:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.541+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f098002d820 con 0x7f0980102850 2024-04-17T07:58:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.542+0000 7f097e000640 1 -- 172.21.15.46:0/1557835917 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f09740f30a0 con 0x7f0980102850 2024-04-17T07:58:48.541 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09680a3bb0 msgr2=0x7f09680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09680a3bb0 0x7f09680a6070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0980197510 tx=0x7f096c005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102850 msgr2=0x7f0980195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102850 0x7f0980195f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09740325f0 tx=0x7f0974035d10 comp rx=0 tx=0).stop 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 shutdown_connections 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09680a3bb0 0x7f09680a6070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0980107ab0 0x7f09801a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980105180 0x7f0980196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 --2- 172.21.15.46:0/1557835917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102850 0x7f0980195f30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.544+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 >> 172.21.15.46:0/1557835917 conn(0x7f09800fc240 msgr2=0x7f0980101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.545+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 shutdown_connections 2024-04-17T07:58:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.545+0000 7f0986c00640 1 -- 172.21.15.46:0/1557835917 wait complete. 2024-04-17T07:58:48.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:58:48.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:48 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3111112715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:48.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:48 smithi046 ceph-mon[32038]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-17T07:58:48.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:48 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1253616640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:48.683 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:48 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3111112715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:48.683 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:48 smithi062 ceph-mon[36507]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-17T07:58:48.683 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:48 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1253616640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:48.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:48 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3111112715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:58:48.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:48 smithi053 ceph-mon[36437]: pgmap v426: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-17T07:58:48.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:48 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1253616640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 -- 172.21.15.46:0/1519234966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01052b0 msgr2=0x7f7ed0105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 --2- 172.21.15.46:0/1519234966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01052b0 0x7f7ed0105730 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009900 tx=0x7f7eb802bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 -- 172.21.15.46:0/1519234966 shutdown_connections 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 --2- 172.21.15.46:0/1519234966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ed0105c70 0x7f7ed010c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 --2- 172.21.15.46:0/1519234966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01052b0 0x7f7ed0105730 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 --2- 172.21.15.46:0/1519234966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed01040c0 0x7f7ed01044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 -- 172.21.15.46:0/1519234966 >> 172.21.15.46:0/1519234966 conn(0x7f7ed00ff630 msgr2=0x7f7ed0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:48.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 -- 172.21.15.46:0/1519234966 shutdown_connections 2024-04-17T07:58:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.786+0000 7f7ed6200640 1 -- 172.21.15.46:0/1519234966 wait complete. 2024-04-17T07:58:48.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 Processor -- start 2024-04-17T07:58:48.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 -- start start 2024-04-17T07:58:48.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01040c0 0x7f7ed019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ed01052b0 0x7f7ed019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0105c70 0x7f7ed01a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010fda0 con 0x7f7ed01040c0 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010fc20 con 0x7f7ed0105c70 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.787+0000 7f7ed6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010ff20 con 0x7f7ed01052b0 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ed4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01040c0 0x7f7ed019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ed01052b0 0x7f7ed019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ed4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01040c0 0x7f7ed019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58240/0 (socket says 172.21.15.46:58240) 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ed4e00640 1 -- 172.21.15.46:0/3618838724 learned_addr learned my addr 172.21.15.46:0/3618838724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ed5800640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0105c70 0x7f7ed01a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ec7e00640 1 -- 172.21.15.46:0/3618838724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0105c70 msgr2=0x7f7ed01a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ec7e00640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0105c70 0x7f7ed01a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ec7e00640 1 -- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01040c0 msgr2=0x7f7ed019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ec7e00640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01040c0 0x7f7ed019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ec7e00640 1 -- 172.21.15.46:0/3618838724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01a1f80 con 0x7f7ed01052b0 2024-04-17T07:58:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.788+0000 7f7ec7e00640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ed01052b0 0x7f7ed019a8a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8036040 tx=0x7f7eb802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.789+0000 7f7ece000640 1 -- 172.21.15.46:0/3618838724 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8041070 con 0x7f7ed01052b0 2024-04-17T07:58:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.789+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01a2210 con 0x7f7ed01052b0 2024-04-17T07:58:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.789+0000 7f7ece000640 1 -- 172.21.15.46:0/3618838724 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb8004450 con 0x7f7ed01052b0 2024-04-17T07:58:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.789+0000 7f7ece000640 1 -- 172.21.15.46:0/3618838724 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8003e60 con 0x7f7ed01052b0 2024-04-17T07:58:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.789+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a2720 con 0x7f7ed01052b0 2024-04-17T07:58:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.790+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed010b380 con 0x7f7ed01052b0 2024-04-17T07:58:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.792+0000 7f7ece000640 1 -- 172.21.15.46:0/3618838724 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7eb8044070 con 0x7f7ed01052b0 2024-04-17T07:58:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.793+0000 7f7ece000640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7eb40a3c20 0x7f7eb40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.794+0000 7f7ece000640 1 -- 172.21.15.46:0/3618838724 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7eb80f2410 con 0x7f7ed01052b0 2024-04-17T07:58:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.794+0000 7f7ed4e00640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7eb40a3c20 0x7f7eb40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.794+0000 7f7ed4e00640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7eb40a3c20 0x7f7eb40a60e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00058c0 tx=0x7f7ec0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:48.796+0000 7f7ece000640 1 -- 172.21.15.46:0/3618838724 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7eb80be3b0 con 0x7f7ed01052b0 2024-04-17T07:58:49.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.047+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed002d5d0 con 0x7f7ed01052b0 2024-04-17T07:58:49.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.048+0000 7f7ece000640 1 -- 172.21.15.46:0/3618838724 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7f7eb80c26b0 con 0x7f7ed01052b0 2024-04-17T07:58:49.048 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:49.048 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:58:39.019766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T07:58:49.048 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-04-17T07:58:49.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7eb40a3c20 msgr2=0x7f7eb40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7eb40a3c20 0x7f7eb40a60e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00058c0 tx=0x7f7ec0005f90 comp rx=0 tx=0).stop 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ed01052b0 msgr2=0x7f7ed019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ed01052b0 0x7f7ed019a8a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8036040 tx=0x7f7eb802c660 comp rx=0 tx=0).stop 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 shutdown_connections 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7eb40a3c20 0x7f7eb40a60e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0105c70 0x7f7ed01a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ed01052b0 0x7f7ed019a8a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 --2- 172.21.15.46:0/3618838724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed01040c0 0x7f7ed019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.051+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 >> 172.21.15.46:0/3618838724 conn(0x7f7ed00ff630 msgr2=0x7f7ed0108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.052+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 shutdown_connections 2024-04-17T07:58:49.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.052+0000 7f7ed6200640 1 -- 172.21.15.46:0/3618838724 wait complete. 2024-04-17T07:58:49.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:58:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 -- 172.21.15.46:0/2550650206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64107b80 msgr2=0x7f9b64109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2550650206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64107b80 0x7f9b64109fd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009be0 tx=0x7f9b5c02bef0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 -- 172.21.15.46:0/2550650206 shutdown_connections 2024-04-17T07:58:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2550650206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64107b80 0x7f9b64109fd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2550650206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b64105250 0x7f9b64107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2550650206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b64102920 0x7f9b64104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 -- 172.21.15.46:0/2550650206 >> 172.21.15.46:0/2550650206 conn(0x7f9b640fc330 msgr2=0x7f9b640fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 -- 172.21.15.46:0/2550650206 shutdown_connections 2024-04-17T07:58:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.346+0000 7f9b69e00640 1 -- 172.21.15.46:0/2550650206 wait complete. 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.347+0000 7f9b69e00640 1 Processor -- start 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69e00640 1 -- start start 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b64102920 0x7f9b6419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64105250 0x7f9b6419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b64107b80 0x7f9b641a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410db70 con 0x7f9b64107b80 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410d9f0 con 0x7f9b64102920 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410dcf0 con 0x7f9b64105250 2024-04-17T07:58:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b68a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b64102920 0x7f9b6419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b64107b80 0x7f9b641a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b5be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64105250 0x7f9b6419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b68a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b64102920 0x7f9b6419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42656/0 (socket says 172.21.15.46:42656) 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.348+0000 7f9b68a00640 1 -- 172.21.15.46:0/2577511311 learned_addr learned my addr 172.21.15.46:0/2577511311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.349+0000 7f9b5be00640 1 -- 172.21.15.46:0/2577511311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b64102920 msgr2=0x7f9b6419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.349+0000 7f9b5be00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b64102920 0x7f9b6419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.349+0000 7f9b5be00640 1 -- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b64107b80 msgr2=0x7f9b641a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.349+0000 7f9b5be00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b64107b80 0x7f9b641a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.349+0000 7f9b5be00640 1 -- 172.21.15.46:0/2577511311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b641a1f60 con 0x7f9b64105250 2024-04-17T07:58:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.349+0000 7f9b5be00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64105250 0x7f9b6419a880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00d6a0 tx=0x7f9b4c00db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.349+0000 7f9b62000640 1 -- 172.21.15.46:0/2577511311 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c017070 con 0x7f9b64105250 2024-04-17T07:58:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.350+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b641a2250 con 0x7f9b64105250 2024-04-17T07:58:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.350+0000 7f9b62000640 1 -- 172.21.15.46:0/2577511311 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4c009820 con 0x7f9b64105250 2024-04-17T07:58:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.350+0000 7f9b62000640 1 -- 172.21.15.46:0/2577511311 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c004cd0 con 0x7f9b64105250 2024-04-17T07:58:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.350+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b641a27f0 con 0x7f9b64105250 2024-04-17T07:58:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.351+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b64109670 con 0x7f9b64105250 2024-04-17T07:58:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.353+0000 7f9b62000640 1 -- 172.21.15.46:0/2577511311 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b4c0099d0 con 0x7f9b64105250 2024-04-17T07:58:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.355+0000 7f9b62000640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9b480a3ba0 0x7f9b480a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.355+0000 7f9b62000640 1 -- 172.21.15.46:0/2577511311 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9b4c0d0090 con 0x7f9b64105250 2024-04-17T07:58:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.355+0000 7f9b68a00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9b480a3ba0 0x7f9b480a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.356+0000 7f9b68a00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9b480a3ba0 0x7f9b480a6060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b540058c0 tx=0x7f9b54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.365+0000 7f9b62000640 1 -- 172.21.15.46:0/2577511311 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b4c09c090 con 0x7f9b64105250 2024-04-17T07:58:49.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.570+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b64062b60 con 0x7f9b64105250 2024-04-17T07:58:49.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.573+0000 7f9b62000640 1 -- 172.21.15.46:0/2577511311 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7f9b4c0a0390 con 0x7f9b64105250 2024-04-17T07:58:49.572 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:58:49.572 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9b480a3ba0 msgr2=0x7f9b480a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9b480a3ba0 0x7f9b480a6060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b540058c0 tx=0x7f9b54009260 comp rx=0 tx=0).stop 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64105250 msgr2=0x7f9b6419a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64105250 0x7f9b6419a880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00d6a0 tx=0x7f9b4c00db60 comp rx=0 tx=0).stop 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 shutdown_connections 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9b480a3ba0 0x7f9b480a6060 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b64107b80 0x7f9b641a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64105250 0x7f9b6419a880 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 --2- 172.21.15.46:0/2577511311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b64102920 0x7f9b6419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.575+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 >> 172.21.15.46:0/2577511311 conn(0x7f9b640fc330 msgr2=0x7f9b64101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.576+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 shutdown_connections 2024-04-17T07:58:49.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.576+0000 7f9b69e00640 1 -- 172.21.15.46:0/2577511311 wait complete. 2024-04-17T07:58:49.593 DEBUG:tasks.fwd_scrub:Yielding 2024-04-17T07:58:49.593 INFO:teuthology.run_tasks:Running task workunit... 2024-04-17T07:58:49.606 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:58:49.607 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-17T07:58:49.607 INFO:tasks.workunit:timeout=3h 2024-04-17T07:58:49.607 INFO:tasks.workunit:cleanup=True 2024-04-17T07:58:49.607 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:49.608 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-17T07:58:49.608 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:58:49.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:58:49.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:58:49.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:49 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1557835917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:49.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:49 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3618838724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-17 07:55:56.596939027 +0000 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-17 07:55:56.596939027 +0000 2024-04-17T07:58:49.656 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-17 07:58:37.923141281 +0000 2024-04-17T07:58:49.657 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-17 07:55:56.596939027 +0000 2024-04-17T07:58:49.657 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-17T07:58:49.657 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-17T07:58:49.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:49 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1557835917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:49.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:49 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3618838724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:49.795 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:58:49.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:49 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1557835917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:58:49.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:49 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3618838724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:58:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 -- 172.21.15.46:0/810290730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c105350 msgr2=0x7fcb7c107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 --2- 172.21.15.46:0/810290730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c105350 0x7fcb7c107740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009900 tx=0x7fcb6802bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 -- 172.21.15.46:0/810290730 shutdown_connections 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 --2- 172.21.15.46:0/810290730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 0x7fcb7c10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 --2- 172.21.15.46:0/810290730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c105350 0x7fcb7c107740 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 --2- 172.21.15.46:0/810290730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb7c102a20 0x7fcb7c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 -- 172.21.15.46:0/810290730 >> 172.21.15.46:0/810290730 conn(0x7fcb7c0fc430 msgr2=0x7fcb7c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 -- 172.21.15.46:0/810290730 shutdown_connections 2024-04-17T07:58:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.824+0000 7fcb82c00640 1 -- 172.21.15.46:0/810290730 wait complete. 2024-04-17T07:58:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 Processor -- start 2024-04-17T07:58:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 -- start start 2024-04-17T07:58:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c102a20 0x7fcb7c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb7c105350 0x7fcb7c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 0x7fcb7c1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c10dc80 con 0x7fcb7c107c80 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c10db00 con 0x7fcb7c105350 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.826+0000 7fcb82c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c10de00 con 0x7fcb7c102a20 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c102a20 0x7fcb7c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 0x7fcb7c1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c102a20 0x7fcb7c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40082/0 (socket says 172.21.15.46:40082) 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 0x7fcb7c1a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58292/0 (socket says 172.21.15.46:58292) 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb7be00640 1 -- 172.21.15.46:0/1196675507 learned_addr learned my addr 172.21.15.46:0/1196675507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb7b400640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb7c105350 0x7fcb7c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 -- 172.21.15.46:0/1196675507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c102a20 msgr2=0x7fcb7c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c102a20 0x7fcb7c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 -- 172.21.15.46:0/1196675507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb7c105350 msgr2=0x7fcb7c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb7c105350 0x7fcb7c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 -- 172.21.15.46:0/1196675507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c1a2180 con 0x7fcb7c107c80 2024-04-17T07:58:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb7be00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c102a20 0x7fcb7c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.827+0000 7fcb80c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 0x7fcb7c1a1b10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009fd0 tx=0x7fcb7400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.828+0000 7fcb78c00640 1 -- 172.21.15.46:0/1196675507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400ce20 con 0x7fcb7c107c80 2024-04-17T07:58:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.828+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c1a2470 con 0x7fcb7c107c80 2024-04-17T07:58:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.828+0000 7fcb78c00640 1 -- 172.21.15.46:0/1196675507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb74005020 con 0x7fcb7c107c80 2024-04-17T07:58:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.828+0000 7fcb78c00640 1 -- 172.21.15.46:0/1196675507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74010720 con 0x7fcb7c107c80 2024-04-17T07:58:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.828+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c1a2940 con 0x7fcb7c107c80 2024-04-17T07:58:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.831+0000 7fcb85976740 1 Processor -- start 2024-04-17T07:58:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.831+0000 7fcb78c00640 1 -- 172.21.15.46:0/1196675507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcb74004a40 con 0x7fcb7c107c80 2024-04-17T07:58:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.831+0000 7fcb85976740 1 -- start start 2024-04-17T07:58:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb85976740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb3ec0 0x560b0ceb42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb85976740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb48b0 0x560b0ceb93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb85976740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cebbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb85976740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b0ce19cb0 con 0x560b0ceb3ec0 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb85976740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560b0cdc0370 con 0x560b0ceb48b0 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb85976740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b0ce15c70 con 0x560b0ceb9a20 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb48b0 0x560b0ceb93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb48b0 0x560b0ceb93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:42688/0 (socket says 172.21.15.46:42688) 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 -- 172.21.15.46:0/3598529719 learned_addr learned my addr 172.21.15.46:0/3598529719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7b400640 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb3ec0 0x560b0ceb42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7b400640 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cebbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 -- 172.21.15.46:0/3598529719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 msgr2=0x560b0cebbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cebbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 -- 172.21.15.46:0/3598529719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb3ec0 msgr2=0x560b0ceb42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb3ec0 0x560b0ceb42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7be00640 1 -- 172.21.15.46:0/3598529719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0cddf910 con 0x560b0ceb48b0 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.832+0000 7fcb7b400640 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cebbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.833+0000 7fcb7be00640 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb48b0 0x560b0ceb93b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c011630 tx=0x7fcb6c011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45ab2ca73b55ab55 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.833+0000 7fcb78c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb600a3bb0 0x7fcb600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.833+0000 7fcb71200640 1 -- 172.21.15.46:0/3598529719 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c01e070 con 0x560b0ceb48b0 2024-04-17T07:58:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.833+0000 7fcb71200640 1 -- 172.21.15.46:0/3598529719 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb6c00fbd0 con 0x560b0ceb48b0 2024-04-17T07:58:49.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.833+0000 7fcb71200640 1 -- 172.21.15.46:0/3598529719 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c019d00 con 0x560b0ceb48b0 2024-04-17T07:58:49.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.833+0000 7fcb78c00640 1 -- 172.21.15.46:0/1196675507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fcb740d0d90 con 0x7fcb7c107c80 2024-04-17T07:58:49.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.833+0000 7fcb7b400640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb600a3bb0 0x7fcb600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 -- 172.21.15.46:0/3598529719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb48b0 msgr2=0x560b0ceb93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb48b0 0x560b0ceb93b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c011630 tx=0x7fcb6c011af0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb7b400640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb600a3bb0 0x7fcb600a6070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcb6802c180 tx=0x7fcb6802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 -- 172.21.15.46:0/3598529719 shutdown_connections 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cebbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb48b0 0x560b0ceb93b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 --2- 172.21.15.46:0/3598529719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb3ec0 0x560b0ceb42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 -- 172.21.15.46:0/3598529719 >> 172.21.15.46:0/3598529719 conn(0x560b0ceaf430 msgr2=0x560b0ceaf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 -- 172.21.15.46:0/3598529719 shutdown_connections 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.834+0000 7fcb85976740 1 -- 172.21.15.46:0/3598529719 wait complete. 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.835+0000 7fcb85976740 1 -- start start 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.835+0000 7fcb85976740 1 Processor -- start 2024-04-17T07:58:49.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb85976740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb3ec0 0x560b0cebe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb85976740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cebe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb85976740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cec5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb85976740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b0cebc9d0 con 0x560b0ceb48b0 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb85976740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560b0cdd3730 con 0x560b0ceb3ec0 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb85976740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b0cebcb50 con 0x560b0ceb9a20 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb85976740 10 client.0 _ll_register_callbacks cb 0x560b0cdd5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cebe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb3ec0 0x560b0cebe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb80c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cec5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cebe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58324/0 (socket says 172.21.15.46:58324) 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb80c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cec5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40098/0 (socket says 172.21.15.46:40098) 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7b400640 1 -- 172.21.15.46:0/2160692063 learned_addr learned my addr 172.21.15.46:0/2160692063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7be00640 1 -- 172.21.15.46:0/2160692063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 msgr2=0x560b0cec5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7be00640 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cec5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7be00640 1 -- 172.21.15.46:0/2160692063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 msgr2=0x560b0cebe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.836+0000 7fcb7be00640 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cebe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.837+0000 7fcb7be00640 1 -- 172.21.15.46:0/2160692063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0cbf47e0 con 0x560b0ceb3ec0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.837+0000 7fcb7be00640 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb3ec0 0x560b0cebe3d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x560b0ceb03a0 tx=0x7fcb6c00fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f9fae503a706a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.837+0000 7fcb65600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.837+0000 7fcb65600640 1 -- 172.21.15.46:0/2160692063 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c026020 con 0x560b0ceb3ec0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.837+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6c00fbd0 con 0x560b0ceb3ec0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.837+0000 7fcb65600640 1 -- 172.21.15.46:0/2160692063 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fcb6c009ce0 con 0x560b0ceb3ec0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb65600640 1 -- 172.21.15.46:0/2160692063 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcb6c031050 con 0x560b0ceb3ec0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 10 client.25556 fetch_fsmap learned FSMap version 24 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560b0cc54f70 con 0x560b0ceb3ec0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb65600640 1 -- 172.21.15.46:0/2160692063 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb6c00fbd0 con 0x560b0ceb3ec0 2024-04-17T07:58:49.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb65600640 1 -- 172.21.15.46:0/2160692063 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fcb6c021070 con 0x560b0ceb3ec0 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 10 client.25556 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 20 client.25556 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cec9f00 0x560b0cecc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 4 client.25556 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.838+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b0ce276d0 con 0x560b0cec9f00 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.839+0000 7fcb7b400640 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cec9f00 0x560b0cecc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.839+0000 7fcb7b400640 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cec9f00 0x560b0cecc370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b6ba13aa0c88a4 server_cookie=64418f7a876261f in_seq=0 out_seq=0 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.839+0000 7fcb65600640 10 client.25556 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb65600640 1 -- 172.21.15.46:0/2160692063 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b0ce276d0 con 0x560b0cec9f00 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb65600640 10 client.25556 handle_command_reply: tid=0 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 client.25556 shutdown 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb3ec0 msgr2=0x560b0cebe3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb3ec0 0x560b0cebe3d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x560b0ceb03a0 tx=0x7fcb6c00fb40 comp rx=0 tx=0).stop 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 shutdown_connections 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cec9f00 0x560b0cecc370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb9a20 0x560b0cec5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cebe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 --2- 172.21.15.46:0/2160692063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb3ec0 0x560b0cebe3d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb65600640 0 client.25556 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 >> 172.21.15.46:0/2160692063 conn(0x560b0ceaf430 msgr2=0x560b0ceae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 shutdown_connections 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.840+0000 7fcb85976740 1 -- 172.21.15.46:0/2160692063 wait complete. 2024-04-17T07:58:49.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.841+0000 7fcb85976740 20 client.25556 trim_cache size 0 max 16384 2024-04-17T07:58:49.846 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 Processor -- start 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 -- start start 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0ceae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0ced1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c012470 con 0x560b0ceb48b0 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c00fd30 con 0x560b0ceb9a20 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb85976740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c00db80 con 0x560b0ceb3ec0 2024-04-17T07:58:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.860+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58330/0 (socket says 172.21.15.46:58330) 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7b400640 1 -- 172.21.15.46:0/2395963588 learned_addr learned my addr 172.21.15.46:0/2395963588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7be00640 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0ceae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb80c00640 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0ced1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7be00640 1 -- 172.21.15.46:0/2395963588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 msgr2=0x560b0ced1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7be00640 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0ced1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7be00640 1 -- 172.21.15.46:0/2395963588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 msgr2=0x560b0cec5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7be00640 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7be00640 1 -- 172.21.15.46:0/2395963588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0caec170 con 0x560b0ceb3ec0 2024-04-17T07:58:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7b400640 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb7be00640 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0ceae3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c010010 tx=0x7fcb6c00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185951cd3eae2888 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.861+0000 7fcb67e00640 1 -- 172.21.15.46:0/2395963588 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c01ae60 con 0x560b0ceb3ec0 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb67e00640 1 -- 172.21.15.46:0/2395963588 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb6c005520 con 0x560b0ceb3ec0 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb67e00640 1 -- 172.21.15.46:0/2395963588 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c026040 con 0x560b0ceb3ec0 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 -- 172.21.15.46:0/2395963588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 msgr2=0x560b0ceae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0ceae3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c010010 tx=0x7fcb6c00f3c0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 -- 172.21.15.46:0/2395963588 shutdown_connections 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0ced1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 --2- 172.21.15.46:0/2395963588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0ceae3c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 -- 172.21.15.46:0/2395963588 >> 172.21.15.46:0/2395963588 conn(0x560b0ceaf430 msgr2=0x560b0ceb2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 -- 172.21.15.46:0/2395963588 shutdown_connections 2024-04-17T07:58:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.862+0000 7fcb85976740 1 -- 172.21.15.46:0/2395963588 wait complete. 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 -- start start 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 Processor -- start 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0cec30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0cec45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c00e1f0 con 0x560b0ceb48b0 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c00f660 con 0x560b0ceb9a20 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c00e8c0 con 0x560b0ceb3ec0 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.863+0000 7fcb85976740 10 client.0 _ll_register_callbacks cb 0x7fcb6c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0cec30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0cec30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40108/0 (socket says 172.21.15.46:40108) 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 -- 172.21.15.46:0/1313168823 learned_addr learned my addr 172.21.15.46:0/1313168823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:49.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7b400640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb80c00640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0cec45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 -- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 msgr2=0x560b0cec45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0cec45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 -- 172.21.15.46:0/1313168823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 msgr2=0x560b0cec3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 -- 172.21.15.46:0/1313168823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0cec4bc0 con 0x560b0ceb3ec0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb80c00640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0cec45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb7be00640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0cec30d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c004ad0 tx=0x7fcb6c004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d9a2c9e53685e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.864+0000 7fcb65600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb65600640 1 -- 172.21.15.46:0/1313168823 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c039070 con 0x560b0ceb3ec0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6c005520 con 0x560b0ceb3ec0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb65600640 1 -- 172.21.15.46:0/1313168823 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fcb6c004c40 con 0x560b0ceb3ec0 2024-04-17T07:58:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb65600640 1 -- 172.21.15.46:0/1313168823 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcb6c045050 con 0x560b0ceb3ec0 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb85976740 10 client.25546 fetch_fsmap learned FSMap version 24 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560b0cf30690 con 0x560b0ceb3ec0 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb65600640 1 -- 172.21.15.46:0/1313168823 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb6c005520 con 0x560b0ceb3ec0 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb65600640 1 -- 172.21.15.46:0/1313168823 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fcb6c021070 con 0x560b0ceb3ec0 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb85976740 10 client.25546 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb85976740 10 client.25546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.865+0000 7fcb85976740 20 client.25546 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.866+0000 7fcb85976740 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cf30f80 0x560b0cf333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.866+0000 7fcb85976740 4 client.25546 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.866+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b0ce21540 con 0x560b0cf30f80 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.866+0000 7fcb7b400640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cf30f80 0x560b0cf333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.866+0000 7fcb7b400640 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cf30f80 0x560b0cf333a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b1d28e7a27262a server_cookie=473c923e9659913c in_seq=0 out_seq=0 2024-04-17T07:58:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.866+0000 7fcb65600640 10 client.25546 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:49.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.867+0000 7fcb65600640 1 -- 172.21.15.46:0/1313168823 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b0ce21540 con 0x560b0cf30f80 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb65600640 10 client.25546 handle_command_reply: tid=0 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 client.25546 shutdown 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 msgr2=0x560b0cec30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0cec30d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c004ad0 tx=0x7fcb6c004990 comp rx=0 tx=0).stop 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 shutdown_connections 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560b0cf30f80 0x560b0cf333a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560b0ceb9a20 0x560b0cec45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b0ceb48b0 0x560b0cec3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb65600640 0 client.25546 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 --2- 172.21.15.46:0/1313168823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b0ceb3ec0 0x560b0cec30d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 >> 172.21.15.46:0/1313168823 conn(0x560b0ceaf430 msgr2=0x560b0ced0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 shutdown_connections 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 1 -- 172.21.15.46:0/1313168823 wait complete. 2024-04-17T07:58:49.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.868+0000 7fcb85976740 20 client.25546 trim_cache size 0 max 16384 2024-04-17T07:58:49.872 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:58:49.872 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-17T07:58:49.872 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4bf00ce3-9cfc-4024-999c-e5f896c03830", 2024-04-17T07:58:49.872 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-17T07:58:49.872 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb600a3bb0 msgr2=0x7fcb600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb600a3bb0 0x7fcb600a6070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcb6802c180 tx=0x7fcb6802e040 comp rx=0 tx=0).stop 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 msgr2=0x7fcb7c1a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 0x7fcb7c1a1b10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009fd0 tx=0x7fcb7400ef10 comp rx=0 tx=0).stop 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 shutdown_connections 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fcb600a3bb0 0x7fcb600a6070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb7c107c80 0x7fcb7c1a1b10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb7c105350 0x7fcb7c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 --2- 172.21.15.46:0/1196675507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb7c102a20 0x7fcb7c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 >> 172.21.15.46:0/1196675507 conn(0x7fcb7c0fc430 msgr2=0x7fcb7c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.869+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 shutdown_connections 2024-04-17T07:58:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:49.870+0000 7fcb82c00640 1 -- 172.21.15.46:0/1196675507 wait complete. 2024-04-17T07:58:49.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:58:49.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:58:50.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 -- 172.21.15.46:0/3868842981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8701031a0 msgr2=0x7fc870103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 --2- 172.21.15.46:0/3868842981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8701031a0 0x7fc870103620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009900 tx=0x7fc85c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 -- 172.21.15.46:0/3868842981 shutdown_connections 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 --2- 172.21.15.46:0/3868842981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc870103b60 0x7fc87010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 --2- 172.21.15.46:0/3868842981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8701031a0 0x7fc870103620 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 --2- 172.21.15.46:0/3868842981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc870101fb0 0x7fc8701023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 -- 172.21.15.46:0/3868842981 >> 172.21.15.46:0/3868842981 conn(0x7fc8700fd520 msgr2=0x7fc8700ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 -- 172.21.15.46:0/3868842981 shutdown_connections 2024-04-17T07:58:50.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.158+0000 7fc876e00640 1 -- 172.21.15.46:0/3868842981 wait complete. 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.159+0000 7fc876e00640 1 Processor -- start 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc876e00640 1 -- start start 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc876e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc870101fb0 0x7fc87019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc876e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8701031a0 0x7fc87019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc876e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc870103b60 0x7fc8701a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc876e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc87010dc90 con 0x7fc870103b60 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc876e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc87010db10 con 0x7fc870101fb0 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc876e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc87010de10 con 0x7fc8701031a0 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc874e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc870103b60 0x7fc8701a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc870101fb0 0x7fc87019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc874e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc870103b60 0x7fc8701a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58358/0 (socket says 172.21.15.46:58358) 2024-04-17T07:58:50.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc874e00640 1 -- 172.21.15.46:0/1913967485 learned_addr learned my addr 172.21.15.46:0/1913967485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.160+0000 7fc86f400640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8701031a0 0x7fc87019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc874e00640 1 -- 172.21.15.46:0/1913967485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8701031a0 msgr2=0x7fc87019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc874e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8701031a0 0x7fc87019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc874e00640 1 -- 172.21.15.46:0/1913967485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc870101fb0 msgr2=0x7fc87019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc874e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc870101fb0 0x7fc87019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc874e00640 1 -- 172.21.15.46:0/1913967485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8701a20d0 con 0x7fc870103b60 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc86fe00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc870101fb0 0x7fc87019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc874e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc870103b60 0x7fc8701a1a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc86800eb10 tx=0x7fc86800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.161+0000 7fc86cc00640 1 -- 172.21.15.46:0/1913967485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868004b20 con 0x7fc870103b60 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.162+0000 7fc86cc00640 1 -- 172.21.15.46:0/1913967485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc868004cc0 con 0x7fc870103b60 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.162+0000 7fc86cc00640 1 -- 172.21.15.46:0/1913967485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868010760 con 0x7fc870103b60 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.162+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8701a23c0 con 0x7fc870103b60 2024-04-17T07:58:50.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.162+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8701a2890 con 0x7fc870103b60 2024-04-17T07:58:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.165+0000 7fc879c3a740 1 Processor -- start 2024-04-17T07:58:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.165+0000 7fc86cc00640 1 -- 172.21.15.46:0/1913967485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc868005020 con 0x7fc870103b60 2024-04-17T07:58:50.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.165+0000 7fc879c3a740 1 -- start start 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1dab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc879c3a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e2a1d12120 con 0x55e2a1dab190 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc879c3a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e2a1cb8340 con 0x55e2a1dabb80 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc879c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e2a1d0e0e0 con 0x55e2a1db0cf0 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc86f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1dab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc86f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1dab5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58374/0 (socket says 172.21.15.46:58374) 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc86f400640 1 -- 172.21.15.46:0/756893556 learned_addr learned my addr 172.21.15.46:0/756893556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc86fe00640 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.166+0000 7fc86f400640 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86fe00640 1 -- 172.21.15.46:0/756893556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 msgr2=0x55e2a1db3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86fe00640 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86fe00640 1 -- 172.21.15.46:0/756893556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 msgr2=0x55e2a1dab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86fe00640 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1dab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86fe00640 1 -- 172.21.15.46:0/756893556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a1aee7d0 con 0x55e2a1dabb80 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86f400640 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1dab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86cc00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8540a3c30 0x7fc8540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86fe00640 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db0680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc860009fa0 tx=0x7fc8600118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c466b2e9536225f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86cc00640 1 -- 172.21.15.46:0/1913967485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc8680d0ce0 con 0x7fc870103b60 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.167+0000 7fc86f400640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8540a3c30 0x7fc8540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.168+0000 7fc865600640 1 -- 172.21.15.46:0/756893556 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860004030 con 0x55e2a1dabb80 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.168+0000 7fc865600640 1 -- 172.21.15.46:0/756893556 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8600041d0 con 0x55e2a1dabb80 2024-04-17T07:58:50.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.168+0000 7fc865600640 1 -- 172.21.15.46:0/756893556 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000f7c0 con 0x55e2a1dabb80 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.168+0000 7fc86f400640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8540a3c30 0x7fc8540a60f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc85c02c720 tx=0x7fc85c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.168+0000 7fc879c3a740 1 -- 172.21.15.46:0/756893556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 msgr2=0x55e2a1db0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.168+0000 7fc879c3a740 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db0680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc860009fa0 tx=0x7fc8600118d0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 -- 172.21.15.46:0/756893556 shutdown_connections 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db0680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 --2- 172.21.15.46:0/756893556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1dab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 -- 172.21.15.46:0/756893556 >> 172.21.15.46:0/756893556 conn(0x55e2a1da6700 msgr2=0x55e2a1da6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 -- 172.21.15.46:0/756893556 shutdown_connections 2024-04-17T07:58:50.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 -- 172.21.15.46:0/756893556 wait complete. 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 -- start start 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.169+0000 7fc879c3a740 1 Processor -- start 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1db56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1dbcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc879c3a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e2a1db3ca0 con 0x55e2a1dab190 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc879c3a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e2a1ccb710 con 0x55e2a1dabb80 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc879c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e2a1db3e20 con 0x55e2a1db0cf0 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc879c3a740 10 client.0 _ll_register_callbacks cb 0x55e2a1cc6af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1db56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1db56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58384/0 (socket says 172.21.15.46:58384) 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc86fe00640 1 -- 172.21.15.46:0/3206448866 learned_addr learned my addr 172.21.15.46:0/3206448866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:50.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.170+0000 7fc86f400640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc874e00640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1dbcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc86fe00640 1 -- 172.21.15.46:0/3206448866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 msgr2=0x55e2a1dbcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc86fe00640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1dbcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc86fe00640 1 -- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 msgr2=0x55e2a1db5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc86fe00640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc86fe00640 1 -- 172.21.15.46:0/3206448866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a1a243f0 con 0x55e2a1dab190 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc874e00640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1dbcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc86f400640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc86fe00640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1db56e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc86001a000 tx=0x7fc860005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82f3d8c4a2a2ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc85a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc85a000640 1 -- 172.21.15.46:0/3206448866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860028020 con 0x55e2a1dab190 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc85a000640 1 -- 172.21.15.46:0/3206448866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc860016040 con 0x55e2a1dab190 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.171+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8600041d0 con 0x55e2a1dab190 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc85a000640 1 -- 172.21.15.46:0/3206448866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc860033050 con 0x55e2a1dab190 2024-04-17T07:58:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e2a19e40f0 con 0x55e2a1dab190 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc85a000640 1 -- 172.21.15.46:0/3206448866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8600041d0 con 0x55e2a1dab190 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc85a000640 1 -- 172.21.15.46:0/3206448866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fc860004b80 con 0x55e2a1dab190 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 20 client.15441 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1dc1350 0x55e2a1dc37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 4 client.15441 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.172+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a1d1fb40 con 0x55e2a1dc1350 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.173+0000 7fc86f400640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1dc1350 0x55e2a1dc37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.174+0000 7fc86f400640 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1dc1350 0x55e2a1dc37a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977602abab7d30a0 server_cookie=868a75243df99df4 in_seq=0 out_seq=0 2024-04-17T07:58:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.174+0000 7fc85a000640 10 client.15441 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.174+0000 7fc85a000640 1 -- 172.21.15.46:0/3206448866 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2a1d1fb40 con 0x55e2a1dc1350 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.174+0000 7fc85a000640 10 client.15441 handle_command_reply: tid=0 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.174+0000 7fc879c3a740 1 client.15441 shutdown 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 msgr2=0x55e2a1db56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1db56e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc86001a000 tx=0x7fc860005d70 comp rx=0 tx=0).stop 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 shutdown_connections 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1dc1350 0x55e2a1dc37a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1dbcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1db5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 --2- 172.21.15.46:0/3206448866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1db56e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 >> 172.21.15.46:0/3206448866 conn(0x55e2a1da6700 msgr2=0x55e2a1da5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc85a000640 0 client.15441 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 shutdown_connections 2024-04-17T07:58:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 1 -- 172.21.15.46:0/3206448866 wait complete. 2024-04-17T07:58:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.175+0000 7fc879c3a740 20 client.15441 trim_cache size 0 max 16384 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 Processor -- start 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 -- start start 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1da1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1dbcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1da5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701af660 con 0x55e2a1dab190 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701acf20 con 0x55e2a1dabb80 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc879c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701aad70 con 0x55e2a1db0cf0 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.195+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1da1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1da1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58386/0 (socket says 172.21.15.46:58386) 2024-04-17T07:58:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 -- 172.21.15.46:0/1963939088 learned_addr learned my addr 172.21.15.46:0/1963939088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86f400640 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1dbcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc874e00640 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1da5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 -- 172.21.15.46:0/1963939088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 msgr2=0x55e2a1da5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1da5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 -- 172.21.15.46:0/1963939088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 msgr2=0x55e2a1dbcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1dbcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 -- 172.21.15.46:0/1963939088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a1da7170 con 0x55e2a1dab190 2024-04-17T07:58:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.196+0000 7fc86fe00640 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1da1720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc860009710 tx=0x7fc860009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe73a50cbb12073 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc864c00640 1 -- 172.21.15.46:0/1963939088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860028070 con 0x55e2a1dab190 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc864c00640 1 -- 172.21.15.46:0/1963939088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc860004040 con 0x55e2a1dab190 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc864c00640 1 -- 172.21.15.46:0/1963939088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860000e60 con 0x55e2a1dab190 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 -- 172.21.15.46:0/1963939088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 msgr2=0x55e2a1da1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1da1720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc860009710 tx=0x7fc860009fa0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 -- 172.21.15.46:0/1963939088 shutdown_connections 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1da5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dabb80 0x55e2a1dbcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 --2- 172.21.15.46:0/1963939088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dab190 0x55e2a1da1720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 -- 172.21.15.46:0/1963939088 >> 172.21.15.46:0/1963939088 conn(0x55e2a1da6700 msgr2=0x55e2a1da5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.197+0000 7fc879c3a740 1 -- 172.21.15.46:0/1963939088 shutdown_connections 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.198+0000 7fc879c3a740 1 -- 172.21.15.46:0/1963939088 wait complete. 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.198+0000 7fc879c3a740 1 -- start start 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.198+0000 7fc879c3a740 1 Processor -- start 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dab190 0x55e2a1dbc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dabb80 0x55e2a1db9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc879c3a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc879c3a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701ab3e0 con 0x55e2a1dabb80 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc879c3a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701ac850 con 0x55e2a1dab190 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc879c3a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701abab0 con 0x55e2a1db0cf0 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc879c3a740 10 client.0 _ll_register_callbacks cb 0x7fc860005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dabb80 0x55e2a1db9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc874e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dabb80 0x55e2a1db9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58402/0 (socket says 172.21.15.46:58402) 2024-04-17T07:58:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 -- 172.21.15.46:0/4081923565 learned_addr learned my addr 172.21.15.46:0/4081923565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86fe00640 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dab190 0x55e2a1dbc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 -- 172.21.15.46:0/4081923565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 msgr2=0x55e2a1db9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 -- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dab190 msgr2=0x55e2a1dbc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dab190 0x55e2a1dbc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86f400640 1 -- 172.21.15.46:0/4081923565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a1db9ed0 con 0x55e2a1dabb80 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.199+0000 7fc86fe00640 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dab190 0x55e2a1dbc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.200+0000 7fc86f400640 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dabb80 0x55e2a1db9400 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009230 tx=0x7fc85c02bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effea45c2dba5ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:58:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.200+0000 7fc85a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:58:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.200+0000 7fc85a000640 1 -- 172.21.15.46:0/4081923565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c004c10 con 0x55e2a1dabb80 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.200+0000 7fc85a000640 1 -- 172.21.15.46:0/4081923565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc85c004ed0 con 0x55e2a1dabb80 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.200+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc860004040 con 0x55e2a1dabb80 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc85a000640 1 -- 172.21.15.46:0/4081923565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc85c067050 con 0x55e2a1dabb80 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 10 client.15447 fetch_fsmap learned FSMap version 24 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e2a1e2f1d0 con 0x55e2a1dabb80 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc85a000640 1 -- 172.21.15.46:0/4081923565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc860004040 con 0x55e2a1dabb80 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc85a000640 1 -- 172.21.15.46:0/4081923565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fc85c003a30 con 0x55e2a1dabb80 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 10 client.15447 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 20 client.15447 populate_metadata read hostname 'smithi046' 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1e2fb60 0x55e2a1e31f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 4 client.15447 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:58:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.201+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2a1d199b0 con 0x55e2a1e2fb60 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.202+0000 7fc86fe00640 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1e2fb60 0x55e2a1e31f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.202+0000 7fc86fe00640 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1e2fb60 0x55e2a1e31f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c89b2f1d6737ca server_cookie=7e4c0aac6b732528 in_seq=0 out_seq=0 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.202+0000 7fc85a000640 10 client.15447 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc85a000640 1 -- 172.21.15.46:0/4081923565 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e2a1d199b0 con 0x55e2a1e2fb60 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc85a000640 10 client.15447 handle_command_reply: tid=0 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 client.15447 shutdown 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dabb80 msgr2=0x55e2a1db9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dabb80 0x55e2a1db9400 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009230 tx=0x7fc85c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 shutdown_connections 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e2a1e2fb60 0x55e2a1e31f80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2a1db0cf0 0x55e2a1db9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc85a000640 0 client.15447 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2a1dabb80 0x55e2a1db9400 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 --2- 172.21.15.46:0/4081923565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2a1dab190 0x55e2a1dbc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 >> 172.21.15.46:0/4081923565 conn(0x55e2a1da6700 msgr2=0x55e2a1da9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 shutdown_connections 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 1 -- 172.21.15.46:0/4081923565 wait complete. 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.203+0000 7fc879c3a740 20 client.15447 trim_cache size 0 max 16384 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "4bf00ce3-9cfc-4024-999c-e5f896c03830": { 2024-04-17T07:58:50.206 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-17T07:58:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4bf00ce3-9cfc-4024-999c-e5f896c03830", 2024-04-17T07:58:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-04-17T07:58:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:58:50.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:58:50.207 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:58:50.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8540a3c30 msgr2=0x7fc8540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8540a3c30 0x7fc8540a60f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc85c02c720 tx=0x7fc85c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc870103b60 msgr2=0x7fc8701a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc870103b60 0x7fc8701a1a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc86800eb10 tx=0x7fc86800c2d0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 shutdown_connections 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc8540a3c30 0x7fc8540a60f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc870103b60 0x7fc8701a1a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8701031a0 0x7fc87019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 --2- 172.21.15.46:0/1913967485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc870101fb0 0x7fc87019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 >> 172.21.15.46:0/1913967485 conn(0x7fc8700fd520 msgr2=0x7fc870074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 shutdown_connections 2024-04-17T07:58:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:58:50.204+0000 7fc876e00640 1 -- 172.21.15.46:0/1913967485 wait complete. 2024-04-17T07:58:50.224 INFO:tasks.cephfs.filesystem:scrub status for tag:4bf00ce3-9cfc-4024-999c-e5f896c03830 - {'path': '/', 'tag': '4bf00ce3-9cfc-4024-999c-e5f896c03830', 'options': 'recursive,force'} 2024-04-17T07:58:50.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:50 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2577511311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:50.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:50 smithi053 ceph-mon[36437]: pgmap v427: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:50.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:50 smithi053 ceph-mon[36437]: scrub queued for path: / 2024-04-17T07:58:50.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:50 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T07:58:50.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:50 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T07:58:50.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:50 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T07:58:50.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:50 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2577511311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:50 smithi062 ceph-mon[36507]: pgmap v427: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:50 smithi062 ceph-mon[36507]: scrub queued for path: / 2024-04-17T07:58:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:50 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T07:58:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:50 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T07:58:50.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:50 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T07:58:50.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:50 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2577511311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:58:50.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:50 smithi046 ceph-mon[32038]: pgmap v427: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:50.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:50 smithi046 ceph-mon[32038]: scrub queued for path: / 2024-04-17T07:58:50.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:50 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T07:58:50.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:50 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T07:58:50.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:50 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T07:58:52.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:52 smithi053 ceph-mon[36437]: pgmap v428: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-17T07:58:53.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:52 smithi062 ceph-mon[36507]: pgmap v428: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-17T07:58:53.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:52 smithi046 ceph-mon[32038]: pgmap v428: 129 pgs: 129 active+clean; 649 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-17T07:58:54.912 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:54 smithi046 ceph-mon[32038]: pgmap v429: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T07:58:54.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:54 smithi053 ceph-mon[36437]: pgmap v429: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T07:58:55.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:54 smithi062 ceph-mon[36507]: pgmap v429: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T07:58:56.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:56 smithi053 ceph-mon[36437]: pgmap v430: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:57.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:56 smithi062 ceph-mon[36507]: pgmap v430: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:57.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:56 smithi046 ceph-mon[32038]: pgmap v430: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:58:58.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:58:58 smithi053 ceph-mon[36437]: pgmap v431: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-17T07:58:59.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:58:58 smithi062 ceph-mon[36507]: pgmap v431: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-17T07:58:59.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:58:58 smithi046 ceph-mon[32038]: pgmap v431: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-17T07:59:00.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:00 smithi053 ceph-mon[36437]: pgmap v432: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:59:00.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:00 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T07:59:01.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:00 smithi062 ceph-mon[36507]: pgmap v432: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:59:01.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:00 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T07:59:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:00 smithi046 ceph-mon[32038]: pgmap v432: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T07:59:01.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:00 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T07:59:02.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:02 smithi053 ceph-mon[36437]: pgmap v433: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-17T07:59:03.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:02 smithi062 ceph-mon[36507]: pgmap v433: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-17T07:59:03.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:02 smithi046 ceph-mon[32038]: pgmap v433: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-17T07:59:04.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:04 smithi053 ceph-mon[36437]: pgmap v434: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:05.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:04 smithi062 ceph-mon[36507]: pgmap v434: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:05.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:04 smithi046 ceph-mon[32038]: pgmap v434: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:06.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:06 smithi053 ceph-mon[36437]: pgmap v435: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T07:59:07.019 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:06 smithi046 ceph-mon[32038]: pgmap v435: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T07:59:07.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:06 smithi062 ceph-mon[36507]: pgmap v435: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T07:59:08.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:08 smithi053 ceph-mon[36437]: pgmap v436: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:09.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:08 smithi062 ceph-mon[36507]: pgmap v436: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:09.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:08 smithi046 ceph-mon[32038]: pgmap v436: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:10.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:10 smithi053 ceph-mon[36437]: pgmap v437: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:11.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:10 smithi062 ceph-mon[36507]: pgmap v437: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:11.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:10 smithi046 ceph-mon[32038]: pgmap v437: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:12.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:12 smithi053 ceph-mon[36437]: pgmap v438: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:59:13.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:12 smithi062 ceph-mon[36507]: pgmap v438: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:59:13.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:12 smithi046 ceph-mon[32038]: pgmap v438: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:59:14.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:14 smithi053 ceph-mon[36437]: pgmap v439: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:15.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:14 smithi062 ceph-mon[36507]: pgmap v439: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:15.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:14 smithi046 ceph-mon[32038]: pgmap v439: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:16.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:16 smithi053 ceph-mon[36437]: pgmap v440: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:17.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:16 smithi062 ceph-mon[36507]: pgmap v440: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:17.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:16 smithi046 ceph-mon[32038]: pgmap v440: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:18.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:18 smithi053 ceph-mon[36437]: pgmap v441: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:19.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:18 smithi062 ceph-mon[36507]: pgmap v441: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:19.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:18 smithi046 ceph-mon[32038]: pgmap v441: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:20.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:59:20.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.470+0000 7f7fea400640 1 -- 172.21.15.46:0/2787709691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 msgr2=0x7f7fe410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.470+0000 7f7fea400640 1 --2- 172.21.15.46:0/2787709691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 0x7f7fe410a230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009900 tx=0x7f7fdc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.471+0000 7f7fea400640 1 -- 172.21.15.46:0/2787709691 shutdown_connections 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.471+0000 7f7fea400640 1 --2- 172.21.15.46:0/2787709691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 0x7f7fe410a230 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.471+0000 7f7fea400640 1 --2- 172.21.15.46:0/2787709691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4102fe0 0x7f7fe4103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.471+0000 7f7fea400640 1 --2- 172.21.15.46:0/2787709691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe4101df0 0x7f7fe4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.471+0000 7f7fea400640 1 -- 172.21.15.46:0/2787709691 >> 172.21.15.46:0/2787709691 conn(0x7f7fe40fd360 msgr2=0x7f7fe40ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.471+0000 7f7fea400640 1 -- 172.21.15.46:0/2787709691 shutdown_connections 2024-04-17T07:59:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.471+0000 7f7fea400640 1 -- 172.21.15.46:0/2787709691 wait complete. 2024-04-17T07:59:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 Processor -- start 2024-04-17T07:59:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 -- start start 2024-04-17T07:59:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4101df0 0x7f7fe419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe4102fe0 0x7f7fe419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 0x7f7fe41a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe410daa0 con 0x7f7fe41039a0 2024-04-17T07:59:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe410d920 con 0x7f7fe4101df0 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.472+0000 7f7fea400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe410dc20 con 0x7f7fe4102fe0 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 0x7f7fe41a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe4102fe0 0x7f7fe419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 0x7f7fe41a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40704/0 (socket says 172.21.15.46:40704) 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe9a00640 1 -- 172.21.15.46:0/4171729613 learned_addr learned my addr 172.21.15.46:0/4171729613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe9000640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4101df0 0x7f7fe419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe3e00640 1 -- 172.21.15.46:0/4171729613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4101df0 msgr2=0x7f7fe419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4101df0 0x7f7fe419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe3e00640 1 -- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 msgr2=0x7f7fe41a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 0x7f7fe41a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe3e00640 1 -- 172.21.15.46:0/4171729613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe41a1f20 con 0x7f7fe4102fe0 2024-04-17T07:59:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.473+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe4102fe0 0x7f7fe419a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000e9f0 tx=0x7f7fd000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.474+0000 7f7fe1600640 1 -- 172.21.15.46:0/4171729613 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000ccc0 con 0x7f7fe4102fe0 2024-04-17T07:59:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.474+0000 7f7fe1600640 1 -- 172.21.15.46:0/4171729613 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd0010910 con 0x7f7fe4102fe0 2024-04-17T07:59:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.474+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe41a2210 con 0x7f7fe4102fe0 2024-04-17T07:59:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.474+0000 7f7fe1600640 1 -- 172.21.15.46:0/4171729613 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0018960 con 0x7f7fe4102fe0 2024-04-17T07:59:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.474+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe41a2760 con 0x7f7fe4102fe0 2024-04-17T07:59:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.477+0000 7f7fee614740 1 Processor -- start 2024-04-17T07:59:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.477+0000 7f7fee614740 1 -- start start 2024-04-17T07:59:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.477+0000 7f7fee614740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23370 0x558bbcf23790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.477+0000 7f7fee614740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf28860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fee614740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf2b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fee614740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558bbce8a120 con 0x558bbcf23d60 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fee614740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558bbce30340 con 0x558bbcf28ed0 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fee614740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558bbce860e0 con 0x558bbcf23370 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe1600640 1 -- 172.21.15.46:0/4171729613 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7fd0010ac0 con 0x7f7fe4102fe0 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23370 0x558bbcf23790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf28860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23370 0x558bbcf23790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50560/0 (socket says 172.21.15.46:50560) 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 -- 172.21.15.46:0/2019341035 learned_addr learned my addr 172.21.15.46:0/2019341035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf2b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 -- 172.21.15.46:0/2019341035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 msgr2=0x558bbcf2b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf2b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 -- 172.21.15.46:0/2019341035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 msgr2=0x558bbcf28860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf28860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9a00640 1 -- 172.21.15.46:0/2019341035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbcc667d0 con 0x558bbcf23370 2024-04-17T07:59:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.478+0000 7f7fe9000640 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf28860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.479+0000 7f7fe9a00640 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23370 0x558bbcf23790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc02c7b0 tx=0x7f7fdc02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5bab252515722c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.479+0000 7f7fda000640 1 -- 172.21.15.46:0/2019341035 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc045070 con 0x558bbcf23370 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.479+0000 7f7fda000640 1 -- 172.21.15.46:0/2019341035 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fdc004040 con 0x558bbcf23370 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.479+0000 7f7fda000640 1 -- 172.21.15.46:0/2019341035 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc004d20 con 0x558bbcf23370 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.479+0000 7f7fe1600640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7fc80a3bb0 0x7f7fc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.479+0000 7f7fe1600640 1 -- 172.21.15.46:0/4171729613 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7fd00d0b30 con 0x7f7fe4102fe0 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fe9000640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7fc80a3bb0 0x7f7fc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 -- 172.21.15.46:0/2019341035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23370 msgr2=0x558bbcf23790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23370 0x558bbcf23790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc02c7b0 tx=0x7f7fdc02c5d0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 -- 172.21.15.46:0/2019341035 shutdown_connections 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf2b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf28860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 --2- 172.21.15.46:0/2019341035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23370 0x558bbcf23790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 -- 172.21.15.46:0/2019341035 >> 172.21.15.46:0/2019341035 conn(0x558bbcf1e8e0 msgr2=0x558bbcf1ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 -- 172.21.15.46:0/2019341035 shutdown_connections 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fee614740 1 -- 172.21.15.46:0/2019341035 wait complete. 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.480+0000 7f7fe9000640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7fc80a3bb0 0x7f7fc80a6070 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009fd0 tx=0x7f7fd4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.481+0000 7f7fee614740 1 -- start start 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.481+0000 7f7fee614740 1 Processor -- start 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.481+0000 7f7fee614740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf23370 0x558bbcf2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.481+0000 7f7fee614740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.481+0000 7f7fee614740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf28ed0 0x558bbcf34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.481+0000 7f7fee614740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558bbcf2be20 con 0x558bbcf23d60 2024-04-17T07:59:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fee614740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558bbce43710 con 0x558bbcf23370 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fee614740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558bbcf2bfa0 con 0x558bbcf28ed0 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fee614740 10 client.0 _ll_register_callbacks cb 0x558bbce3eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf2ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40716/0 (socket says 172.21.15.46:40716) 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 -- 172.21.15.46:0/2230692276 learned_addr learned my addr 172.21.15.46:0/2230692276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe9a00640 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf28ed0 0x558bbcf34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe9000640 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf23370 0x558bbcf2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 -- 172.21.15.46:0/2230692276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf28ed0 msgr2=0x558bbcf34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf28ed0 0x558bbcf34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 -- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf23370 msgr2=0x558bbcf2d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf23370 0x558bbcf2d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.482+0000 7f7fe3e00640 1 -- 172.21.15.46:0/2230692276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbcb9c3f0 con 0x558bbcf23d60 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf2ddd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00a5160 tx=0x7f7fd009c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83122a812f029e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fcea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fcea00640 1 -- 172.21.15.46:0/2230692276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd00bbde0 con 0x558bbcf23d60 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fcea00640 1 -- 172.21.15.46:0/2230692276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7fd009bf20 con 0x558bbcf23d60 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fdc004040 con 0x558bbcf23d60 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fcea00640 1 -- 172.21.15.46:0/2230692276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7fd00bd070 con 0x558bbcf23d60 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fee614740 10 client.15459 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558bbcb5c0f0 con 0x558bbcf23d60 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.483+0000 7f7fcea00640 1 -- 172.21.15.46:0/2230692276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fdc004040 con 0x558bbcf23d60 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fcea00640 1 -- 172.21.15.46:0/2230692276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f7fd0018b90 con 0x558bbcf23d60 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fee614740 10 client.15459 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fee614740 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fee614740 20 client.15459 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fee614740 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcf39600 0x558bbcf3baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fee614740 4 client.15459 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bbce97b40 con 0x558bbcf39600 2024-04-17T07:59:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.484+0000 7f7fe9000640 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcf39600 0x558bbcf3baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.485+0000 7f7fe9000640 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcf39600 0x558bbcf3baa0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0aebf927e9a27c server_cookie=4829e59c4c9c83b6 in_seq=0 out_seq=0 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.485+0000 7f7fcea00640 10 client.15459 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.485+0000 7f7fcea00640 1 -- 172.21.15.46:0/2230692276 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bbce97b40 con 0x558bbcf39600 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.485+0000 7f7fcea00640 10 client.15459 handle_command_reply: tid=0 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 client.15459 shutdown 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 msgr2=0x558bbcf2ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf2ddd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00a5160 tx=0x7f7fd009c610 comp rx=0 tx=0).stop 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 shutdown_connections 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcf39600 0x558bbcf3baa0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf28ed0 0x558bbcf34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23d60 0x558bbcf2ddd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 --2- 172.21.15.46:0/2230692276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf23370 0x558bbcf2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fcea00640 0 client.15459 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 >> 172.21.15.46:0/2230692276 conn(0x558bbcf1e8e0 msgr2=0x558bbcf1dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 shutdown_connections 2024-04-17T07:59:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 1 -- 172.21.15.46:0/2230692276 wait complete. 2024-04-17T07:59:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.486+0000 7f7fee614740 20 client.15459 trim_cache size 0 max 16384 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.505+0000 7f7fee614740 1 Processor -- start 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fee614740 1 -- start start 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fee614740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf22250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fee614740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf34d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fee614740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fee614740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41af530 con 0x558bbcf23370 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fee614740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41acdf0 con 0x558bbcf28ed0 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fee614740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41aac40 con 0x558bbcf23d60 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fe9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf22250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fe9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf22250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40720/0 (socket says 172.21.15.46:40720) 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fe9000640 1 -- 172.21.15.46:0/1413336353 learned_addr learned my addr 172.21.15.46:0/1413336353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:20.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf34d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.506+0000 7f7fe9a00640 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fe9000640 1 -- 172.21.15.46:0/1413336353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 msgr2=0x558bbcf34d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fe9000640 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf34d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fe9000640 1 -- 172.21.15.46:0/1413336353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 msgr2=0x558bbcf40360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fe9000640 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf40360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fe9000640 1 -- 172.21.15.46:0/1413336353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbcf245f0 con 0x558bbcf23370 2024-04-17T07:59:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fe9000640 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf22250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4005fd0 tx=0x7f7fd401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84839bb9a6e5c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fd9600640 1 -- 172.21.15.46:0/1413336353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400fda0 con 0x558bbcf23370 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fd9600640 1 -- 172.21.15.46:0/1413336353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd402a430 con 0x558bbcf23370 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.507+0000 7f7fd9600640 1 -- 172.21.15.46:0/1413336353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40054f0 con 0x558bbcf23370 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 -- 172.21.15.46:0/1413336353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 msgr2=0x558bbcf22250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf22250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4005fd0 tx=0x7f7fd401a000 comp rx=0 tx=0).stop 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 -- 172.21.15.46:0/1413336353 shutdown_connections 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf40360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf34d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 --2- 172.21.15.46:0/1413336353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf22250 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 -- 172.21.15.46:0/1413336353 >> 172.21.15.46:0/1413336353 conn(0x558bbcf1e8e0 msgr2=0x558bbcf21b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 -- 172.21.15.46:0/1413336353 shutdown_connections 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 -- 172.21.15.46:0/1413336353 wait complete. 2024-04-17T07:59:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.508+0000 7f7fee614740 1 -- start start 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 1 Processor -- start 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf33230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41ab2b0 con 0x558bbcf23370 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41ac720 con 0x558bbcf28ed0 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41ab980 con 0x558bbcf23d60 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fee614740 10 client.0 _ll_register_callbacks cb 0x7f7fd000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fe9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fe9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf32cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40734/0 (socket says 172.21.15.46:40734) 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fe9000640 1 -- 172.21.15.46:0/709725834 learned_addr learned my addr 172.21.15.46:0/709725834 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.509+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf33230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fe9a00640 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fe9000640 1 -- 172.21.15.46:0/709725834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 msgr2=0x558bbcf33230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fe9000640 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf33230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fe9000640 1 -- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 msgr2=0x558bbcf341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fe9000640 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fe9000640 1 -- 172.21.15.46:0/709725834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbcf347c0 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fe9000640 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf32cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009810 tx=0x7f7fd4012130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf0003db98246ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.510+0000 7f7fcea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fcea00640 1 -- 172.21.15.46:0/709725834 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4036020 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fcea00640 1 -- 172.21.15.46:0/709725834 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7fd402a5e0 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd402a430 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fcea00640 1 -- 172.21.15.46:0/709725834 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7fd402a7e0 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fee614740 10 client.15465 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558bbcfa0410 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fcea00640 1 -- 172.21.15.46:0/709725834 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd402a430 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fcea00640 1 -- 172.21.15.46:0/709725834 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f7fd4004a40 con 0x558bbcf23370 2024-04-17T07:59:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fee614740 10 client.15465 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fee614740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.511+0000 7f7fee614740 20 client.15465 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.512+0000 7f7fee614740 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcfa0e60 0x558bbcfa3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.512+0000 7f7fee614740 4 client.15465 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:59:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.512+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558bbce919b0 con 0x558bbcfa0e60 2024-04-17T07:59:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.512+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcfa0e60 0x558bbcfa3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.512+0000 7f7fe3e00640 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcfa0e60 0x558bbcfa3280 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5e2d9e1a92afa3 server_cookie=2c42c47c75d93230 in_seq=0 out_seq=0 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.512+0000 7f7fcea00640 10 client.15465 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fcea00640 1 -- 172.21.15.46:0/709725834 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558bbce919b0 con 0x558bbcfa0e60 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fcea00640 10 client.15465 handle_command_reply: tid=0 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 client.15465 shutdown 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 msgr2=0x558bbcf32cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf32cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009810 tx=0x7f7fd4012130 comp rx=0 tx=0).stop 2024-04-17T07:59:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 shutdown_connections 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558bbcfa0e60 0x558bbcfa3280 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bbcf28ed0 0x558bbcf341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558bbcf23d60 0x558bbcf33230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 --2- 172.21.15.46:0/709725834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bbcf23370 0x558bbcf32cf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fcea00640 0 client.15465 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 >> 172.21.15.46:0/709725834 conn(0x558bbcf1e8e0 msgr2=0x558bbcf223d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 shutdown_connections 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 1 -- 172.21.15.46:0/709725834 wait complete. 2024-04-17T07:59:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.513+0000 7f7fee614740 20 client.15465 trim_cache size 0 max 16384 2024-04-17T07:59:20.517 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:59:20.517 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-17T07:59:20.517 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-17T07:59:20.517 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:59:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7fc80a3bb0 msgr2=0x7f7fc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7fc80a3bb0 0x7f7fc80a6070 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009fd0 tx=0x7f7fd4009500 comp rx=0 tx=0).stop 2024-04-17T07:59:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe4102fe0 msgr2=0x7f7fe419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe4102fe0 0x7f7fe419a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000e9f0 tx=0x7f7fd000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 shutdown_connections 2024-04-17T07:59:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7fc80a3bb0 0x7f7fc80a6070 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fe41039a0 0x7f7fe41a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe4102fe0 0x7f7fe419a840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 --2- 172.21.15.46:0/4171729613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe4101df0 0x7f7fe419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 >> 172.21.15.46:0/4171729613 conn(0x7f7fe40fd360 msgr2=0x7f7fe4074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 shutdown_connections 2024-04-17T07:59:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.514+0000 7f7fea400640 1 -- 172.21.15.46:0/4171729613 wait complete. 2024-04-17T07:59:20.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:59:20.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.755+0000 7f5d18400640 1 -- 172.21.15.46:0/3586727401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1010a190 msgr2=0x7f5d10110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.755+0000 7f5d18400640 1 --2- 172.21.15.46:0/3586727401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1010a190 0x7f5d10110a20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009ca0 tx=0x7f5d0802c100 comp rx=0 tx=0).stop 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.755+0000 7f5d18400640 1 -- 172.21.15.46:0/3586727401 shutdown_connections 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.755+0000 7f5d18400640 1 --2- 172.21.15.46:0/3586727401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1010a190 0x7f5d10110a20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.755+0000 7f5d18400640 1 --2- 172.21.15.46:0/3586727401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d101097d0 0x7f5d10109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.755+0000 7f5d18400640 1 --2- 172.21.15.46:0/3586727401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101085e0 0x7f5d10108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.755+0000 7f5d18400640 1 -- 172.21.15.46:0/3586727401 >> 172.21.15.46:0/3586727401 conn(0x7f5d100775b0 msgr2=0x7f5d100799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.756+0000 7f5d18400640 1 -- 172.21.15.46:0/3586727401 shutdown_connections 2024-04-17T07:59:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.756+0000 7f5d18400640 1 -- 172.21.15.46:0/3586727401 wait complete. 2024-04-17T07:59:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 Processor -- start 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 -- start start 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101085e0 0x7f5d1019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d101097d0 0x7f5d1019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1010a190 0x7f5d101a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10114240 con 0x7f5d101097d0 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101140c0 con 0x7f5d101085e0 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d18400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101143c0 con 0x7f5d1010a190 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d15000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d101097d0 0x7f5d1019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.757+0000 7f5d15000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d101097d0 0x7f5d1019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40750/0 (socket says 172.21.15.46:40750) 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15000640 1 -- 172.21.15.46:0/315037148 learned_addr learned my addr 172.21.15.46:0/315037148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d16400640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1010a190 0x7f5d101a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15a00640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101085e0 0x7f5d1019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15000640 1 -- 172.21.15.46:0/315037148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1010a190 msgr2=0x7f5d101a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15000640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1010a190 0x7f5d101a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15000640 1 -- 172.21.15.46:0/315037148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101085e0 msgr2=0x7f5d1019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15000640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101085e0 0x7f5d1019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15000640 1 -- 172.21.15.46:0/315037148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d101a6470 con 0x7f5d101097d0 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d15000640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d101097d0 0x7f5d1019ed90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000ea60 tx=0x7f5d0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.758+0000 7f5d0e000640 1 -- 172.21.15.46:0/315037148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000cde0 con 0x7f5d101097d0 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.759+0000 7f5d0e000640 1 -- 172.21.15.46:0/315037148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d000035f0 con 0x7f5d101097d0 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.759+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d101a6760 con 0x7f5d101097d0 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.759+0000 7f5d0e000640 1 -- 172.21.15.46:0/315037148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00010670 con 0x7f5d101097d0 2024-04-17T07:59:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.759+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d101a6cb0 con 0x7f5d101097d0 2024-04-17T07:59:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.760+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d1010f900 con 0x7f5d101097d0 2024-04-17T07:59:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.762+0000 7f5d0e000640 1 -- 172.21.15.46:0/315037148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5d000037a0 con 0x7f5d101097d0 2024-04-17T07:59:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.763+0000 7f5d0e000640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5cf80a3bb0 0x7f5cf80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.764+0000 7f5d0e000640 1 -- 172.21.15.46:0/315037148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f5d000d0aa0 con 0x7f5d101097d0 2024-04-17T07:59:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.764+0000 7f5d15a00640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5cf80a3bb0 0x7f5cf80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.765+0000 7f5d15a00640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5cf80a3bb0 0x7f5cf80a6070 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d10109630 tx=0x7f5d04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:20.769+0000 7f5d0e000640 1 -- 172.21.15.46:0/315037148 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5d0009d090 con 0x7f5d101097d0 2024-04-17T07:59:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.017+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d1002d5d0 con 0x7f5d101097d0 2024-04-17T07:59:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.020+0000 7f5d0e000640 1 -- 172.21.15.46:0/315037148 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7f5d0009c990 con 0x7f5d101097d0 2024-04-17T07:59:21.019 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:59:21.019 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:58:39.019766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T07:59:21.019 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-04-17T07:59:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.022+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5cf80a3bb0 msgr2=0x7f5cf80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.022+0000 7f5d18400640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5cf80a3bb0 0x7f5cf80a6070 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d10109630 tx=0x7f5d04005f90 comp rx=0 tx=0).stop 2024-04-17T07:59:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.022+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d101097d0 msgr2=0x7f5d1019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.022+0000 7f5d18400640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d101097d0 0x7f5d1019ed90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000ea60 tx=0x7f5d0000ef20 comp rx=0 tx=0).stop 2024-04-17T07:59:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 shutdown_connections 2024-04-17T07:59:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f5cf80a3bb0 0x7f5cf80a6070 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d1010a190 0x7f5d101a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d101097d0 0x7f5d1019ed90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 --2- 172.21.15.46:0/315037148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101085e0 0x7f5d1019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 >> 172.21.15.46:0/315037148 conn(0x7f5d100775b0 msgr2=0x7f5d1010cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 shutdown_connections 2024-04-17T07:59:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.023+0000 7f5d18400640 1 -- 172.21.15.46:0/315037148 wait complete. 2024-04-17T07:59:21.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:20 smithi046 ceph-mon[32038]: pgmap v442: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-17T07:59:21.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:59:21.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:20 smithi062 ceph-mon[36507]: pgmap v442: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-17T07:59:21.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:20 smithi053 ceph-mon[36437]: pgmap v442: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-17T07:59:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 -- 172.21.15.46:0/1870862819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24103fe0 msgr2=0x7ffa24104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 --2- 172.21.15.46:0/1870862819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24103fe0 0x7ffa24104400 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009900 tx=0x7ffa1402bc40 comp rx=0 tx=0).stop 2024-04-17T07:59:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 -- 172.21.15.46:0/1870862819 shutdown_connections 2024-04-17T07:59:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 --2- 172.21.15.46:0/1870862819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa24105b90 0x7ffa2410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 --2- 172.21.15.46:0/1870862819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa241051d0 0x7ffa24105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 --2- 172.21.15.46:0/1870862819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24103fe0 0x7ffa24104400 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 -- 172.21.15.46:0/1870862819 >> 172.21.15.46:0/1870862819 conn(0x7ffa240ff570 msgr2=0x7ffa24101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 -- 172.21.15.46:0/1870862819 shutdown_connections 2024-04-17T07:59:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.272+0000 7ffa23200640 1 -- 172.21.15.46:0/1870862819 wait complete. 2024-04-17T07:59:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.273+0000 7ffa23200640 1 Processor -- start 2024-04-17T07:59:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.273+0000 7ffa23200640 1 -- start start 2024-04-17T07:59:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa23200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa24103fe0 0x7ffa2419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa23200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa241051d0 0x7ffa2419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa23200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24105b90 0x7ffa241a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa23200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2410fc70 con 0x7ffa24103fe0 2024-04-17T07:59:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa23200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2410faf0 con 0x7ffa24105b90 2024-04-17T07:59:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa23200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2410fdf0 con 0x7ffa241051d0 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa24103fe0 0x7ffa2419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa24103fe0 0x7ffa2419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40764/0 (socket says 172.21.15.46:40764) 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa21e00640 1 -- 172.21.15.46:0/3386903643 learned_addr learned my addr 172.21.15.46:0/3386903643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa22800640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24105b90 0x7ffa241a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.274+0000 7ffa21400640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa241051d0 0x7ffa2419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa22800640 1 -- 172.21.15.46:0/3386903643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa241051d0 msgr2=0x7ffa2419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa22800640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa241051d0 0x7ffa2419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa22800640 1 -- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa24103fe0 msgr2=0x7ffa2419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa22800640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa24103fe0 0x7ffa2419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa22800640 1 -- 172.21.15.46:0/3386903643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa241a1f70 con 0x7ffa24105b90 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa21e00640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa24103fe0 0x7ffa2419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa22800640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24105b90 0x7ffa241a1900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009fd0 tx=0x7ffa1c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa1aa00640 1 -- 172.21.15.46:0/3386903643 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00ce20 con 0x7ffa24105b90 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa241a2260 con 0x7ffa24105b90 2024-04-17T07:59:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa1aa00640 1 -- 172.21.15.46:0/3386903643 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c005020 con 0x7ffa24105b90 2024-04-17T07:59:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.275+0000 7ffa1aa00640 1 -- 172.21.15.46:0/3386903643 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c010720 con 0x7ffa24105b90 2024-04-17T07:59:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.276+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa241a27b0 con 0x7ffa24105b90 2024-04-17T07:59:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.279+0000 7ffa2ba51740 1 Processor -- start 2024-04-17T07:59:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.279+0000 7ffa1aa00640 1 -- 172.21.15.46:0/3386903643 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffa1c005540 con 0x7ffa24105b90 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.279+0000 7ffa2ba51740 1 -- start start 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0adc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0ae1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0ae4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5605a0a42020 con 0x5605a0adcb50 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5605a09e8460 con 0x5605a0adc160 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5605a0a3dfe0 con 0x5605a0ae1cc0 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0adc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0adc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32862/0 (socket says 172.21.15.46:32862) 2024-04-17T07:59:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0ae1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21400640 1 -- 172.21.15.46:0/3850233423 learned_addr learned my addr 172.21.15.46:0/3850233423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21400640 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0ae4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21e00640 1 -- 172.21.15.46:0/3850233423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 msgr2=0x5605a0ae4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21e00640 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0ae4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.280+0000 7ffa21e00640 1 -- 172.21.15.46:0/3850233423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 msgr2=0x5605a0adc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa21e00640 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0adc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa21e00640 1 -- 172.21.15.46:0/3850233423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a09f0040 con 0x5605a0adcb50 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa21400640 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0adc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa1aa00640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffa080a3c30 0x7ffa080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa1aa00640 1 -- 172.21.15.46:0/3386903643 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ffa1c0d0e70 con 0x7ffa24105b90 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa21e00640 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0ae1650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa1402c7c0 tx=0x7ffa1402cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b25fd8c49bc179 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa21400640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffa080a3c30 0x7ffa080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa0f400640 1 -- 172.21.15.46:0/3850233423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14042070 con 0x5605a0adcb50 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa0f400640 1 -- 172.21.15.46:0/3850233423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa14004ba0 con 0x5605a0adcb50 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.281+0000 7ffa0f400640 1 -- 172.21.15.46:0/3850233423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14005110 con 0x5605a0adcb50 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 -- 172.21.15.46:0/3850233423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 msgr2=0x5605a0ae1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa21400640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffa080a3c30 0x7ffa080a60f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5605a0add3b0 tx=0x7ffa10009370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0ae1650 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa1402c7c0 tx=0x7ffa1402cd40 comp rx=0 tx=0).stop 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 -- 172.21.15.46:0/3850233423 shutdown_connections 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0ae4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0ae1650 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/3850233423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0adc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 -- 172.21.15.46:0/3850233423 >> 172.21.15.46:0/3850233423 conn(0x5605a0ad76d0 msgr2=0x5605a0ad7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 -- 172.21.15.46:0/3850233423 shutdown_connections 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.282+0000 7ffa2ba51740 1 -- 172.21.15.46:0/3850233423 wait complete. 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 -- start start 2024-04-17T07:59:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 Processor -- start 2024-04-17T07:59:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ae6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0adcb50 0x5605a0ae6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0ae1cc0 0x5605a0aedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5605a0ae4c10 con 0x5605a0ae1cc0 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5605a09fb830 con 0x5605a0adc160 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.283+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5605a0ae4d90 con 0x5605a0adcb50 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa2ba51740 10 client.0 _ll_register_callbacks cb 0x5605a09c9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ae6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0ae1cc0 0x5605a0aedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ae6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32876/0 (socket says 172.21.15.46:32876) 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 -- 172.21.15.46:0/1718721608 learned_addr learned my addr 172.21.15.46:0/1718721608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21400640 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0adcb50 0x5605a0ae6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 -- 172.21.15.46:0/1718721608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0adcb50 msgr2=0x5605a0ae6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0adcb50 0x5605a0ae6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 -- 172.21.15.46:0/1718721608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0ae1cc0 msgr2=0x5605a0aedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0ae1cc0 0x5605a0aedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 -- 172.21.15.46:0/1718721608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a09ff920 con 0x5605a0adc160 2024-04-17T07:59:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa21e00640 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ae6640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa14006010 tx=0x7ffa14004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46ec7bcd83e39e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.284+0000 7ffa03e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.285+0000 7ffa03e00640 1 -- 172.21.15.46:0/1718721608 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14042070 con 0x5605a0adc160 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.285+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa14004ba0 con 0x5605a0adc160 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.285+0000 7ffa03e00640 1 -- 172.21.15.46:0/1718721608 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ffa140052e0 con 0x5605a0adc160 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.285+0000 7ffa03e00640 1 -- 172.21.15.46:0/1718721608 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffa14057050 con 0x5605a0adc160 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.285+0000 7ffa2ba51740 10 client.25616 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.285+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5605a09cd7f0 con 0x5605a0adc160 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.285+0000 7ffa03e00640 1 -- 172.21.15.46:0/1718721608 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa14004ba0 con 0x5605a0adc160 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa03e00640 1 -- 172.21.15.46:0/1718721608 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7ffa14034440 con 0x5605a0adc160 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa2ba51740 10 client.25616 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa2ba51740 10 client.25616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa2ba51740 20 client.25616 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0af1d20 0x5605a0af4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa2ba51740 4 client.25616 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605a0a4fa40 con 0x5605a0af1d20 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.286+0000 7ffa21400640 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0af1d20 0x5605a0af4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.287+0000 7ffa21400640 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0af1d20 0x5605a0af4110 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=667d9ad7e68cfbd0 server_cookie=186b91831a8e7391 in_seq=0 out_seq=0 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.287+0000 7ffa03e00640 10 client.25616 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa03e00640 1 -- 172.21.15.46:0/1718721608 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605a0a4fa40 con 0x5605a0af1d20 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa03e00640 10 client.25616 handle_command_reply: tid=0 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 client.25616 shutdown 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 msgr2=0x5605a0ae6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ae6640 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa14006010 tx=0x7ffa14004970 comp rx=0 tx=0).stop 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 shutdown_connections 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0af1d20 0x5605a0af4110 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0ae1cc0 0x5605a0aedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0adcb50 0x5605a0ae6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa03e00640 0 client.25616 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/1718721608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ae6640 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 >> 172.21.15.46:0/1718721608 conn(0x5605a0ad76d0 msgr2=0x5605a0ad6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 shutdown_connections 2024-04-17T07:59:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 1 -- 172.21.15.46:0/1718721608 wait complete. 2024-04-17T07:59:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.288+0000 7ffa2ba51740 20 client.25616 trim_cache size 0 max 16384 2024-04-17T07:59:21.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.303+0000 7ffa2ba51740 1 Processor -- start 2024-04-17T07:59:21.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.303+0000 7ffa2ba51740 1 -- start start 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ad67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0aedae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0af8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241af3c0 con 0x5605a0adcb50 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241acc80 con 0x5605a0adc160 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241aaad0 con 0x5605a0ae1cc0 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0aedae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ad67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ad67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32880/0 (socket says 172.21.15.46:32880) 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21e00640 1 -- 172.21.15.46:0/139043248 learned_addr learned my addr 172.21.15.46:0/139043248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa22800640 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0af8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21e00640 1 -- 172.21.15.46:0/139043248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 msgr2=0x5605a0af8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21e00640 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0af8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21e00640 1 -- 172.21.15.46:0/139043248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 msgr2=0x5605a0aedae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.304+0000 7ffa21e00640 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0aedae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.305+0000 7ffa21e00640 1 -- 172.21.15.46:0/139043248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a07543f0 con 0x5605a0adc160 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.305+0000 7ffa21400640 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0aedae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.305+0000 7ffa22800640 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0af8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.305+0000 7ffa21e00640 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ad67b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa14034a50 tx=0x7ffa14005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feae4e0ad056078e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.305+0000 7ffa0ea00640 1 -- 172.21.15.46:0/139043248 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14060070 con 0x5605a0adc160 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.305+0000 7ffa0ea00640 1 -- 172.21.15.46:0/139043248 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1404e070 con 0x5605a0adc160 2024-04-17T07:59:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.305+0000 7ffa0ea00640 1 -- 172.21.15.46:0/139043248 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14042050 con 0x5605a0adc160 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 -- 172.21.15.46:0/139043248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 msgr2=0x5605a0ad67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ad67b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa14034a50 tx=0x7ffa14005b50 comp rx=0 tx=0).stop 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 -- 172.21.15.46:0/139043248 shutdown_connections 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0af8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0adcb50 0x5605a0aedae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/139043248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adc160 0x5605a0ad67b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 -- 172.21.15.46:0/139043248 >> 172.21.15.46:0/139043248 conn(0x5605a0ad76d0 msgr2=0x5605a0ad60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 -- 172.21.15.46:0/139043248 shutdown_connections 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.306+0000 7ffa2ba51740 1 -- 172.21.15.46:0/139043248 wait complete. 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 -- start start 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 Processor -- start 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adcb50 0x5605a0aebad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0aec010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0aecf80 0x5605a0aed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241ab140 con 0x5605a0aecf80 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241ac5b0 con 0x5605a0adcb50 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241ab810 con 0x5605a0ae1cc0 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa2ba51740 10 client.0 _ll_register_callbacks cb 0x7ffa14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.307+0000 7ffa22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0aecf80 0x5605a0aed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0aec010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adcb50 0x5605a0aebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0aec010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50670/0 (socket says 172.21.15.46:50670) 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 -- 172.21.15.46:0/866801744 learned_addr learned my addr 172.21.15.46:0/866801744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0aecf80 0x5605a0aed400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40814/0 (socket says 172.21.15.46:40814) 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 -- 172.21.15.46:0/866801744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adcb50 msgr2=0x5605a0aebad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adcb50 0x5605a0aebad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 -- 172.21.15.46:0/866801744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0aecf80 msgr2=0x5605a0aed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0aecf80 0x5605a0aed400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 -- 172.21.15.46:0/866801744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a081e7d0 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa21400640 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0aec010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5605a0af48e0 tx=0x7ffa10004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f71adac1abb851af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa03e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa03e00640 1 -- 172.21.15.46:0/866801744 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10004490 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1404e070 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.308+0000 7ffa03e00640 1 -- 172.21.15.46:0/866801744 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ffa1001a070 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa03e00640 1 -- 172.21.15.46:0/866801744 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffa10036050 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 10 client.25606 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5605a0b61520 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa03e00640 1 -- 172.21.15.46:0/866801744 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1404e070 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa03e00640 1 -- 172.21.15.46:0/866801744 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7ffa1001d070 con 0x5605a0ae1cc0 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 10 client.25606 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 10 client.25606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T07:59:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 20 client.25606 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0b61f00 0x5605a0b64320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 4 client.25606 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:59:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.309+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605a0a498b0 con 0x5605a0b61f00 2024-04-17T07:59:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.310+0000 7ffa21e00640 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0b61f00 0x5605a0b64320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.310+0000 7ffa21e00640 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0b61f00 0x5605a0b64320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdce720c22456e server_cookie=20d306c47976e59f in_seq=0 out_seq=0 2024-04-17T07:59:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.310+0000 7ffa03e00640 10 client.25606 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.310+0000 7ffa03e00640 1 -- 172.21.15.46:0/866801744 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605a0a498b0 con 0x5605a0b61f00 2024-04-17T07:59:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.310+0000 7ffa03e00640 10 client.25606 handle_command_reply: tid=0 2024-04-17T07:59:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.310+0000 7ffa2ba51740 1 client.25606 shutdown 2024-04-17T07:59:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 msgr2=0x5605a0aec010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0aec010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5605a0af48e0 tx=0x7ffa10004730 comp rx=0 tx=0).stop 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 shutdown_connections 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5605a0b61f00 0x5605a0b64320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605a0aecf80 0x5605a0aed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a0ae1cc0 0x5605a0aec010 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 --2- 172.21.15.46:0/866801744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605a0adcb50 0x5605a0aebad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa03e00640 0 client.25606 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 >> 172.21.15.46:0/866801744 conn(0x5605a0ad76d0 msgr2=0x5605a0ad6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 shutdown_connections 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 1 -- 172.21.15.46:0/866801744 wait complete. 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.311+0000 7ffa2ba51740 20 client.25606 trim_cache size 0 max 16384 2024-04-17T07:59:21.314 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T07:59:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffa080a3c30 msgr2=0x7ffa080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffa080a3c30 0x7ffa080a60f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x5605a0add3b0 tx=0x7ffa10009370 comp rx=0 tx=0).stop 2024-04-17T07:59:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24105b90 msgr2=0x7ffa241a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24105b90 0x7ffa241a1900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009fd0 tx=0x7ffa1c00ef10 comp rx=0 tx=0).stop 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 shutdown_connections 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ffa080a3c30 0x7ffa080a60f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa24105b90 0x7ffa241a1900 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa241051d0 0x7ffa2419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 --2- 172.21.15.46:0/3386903643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa24103fe0 0x7ffa2419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 >> 172.21.15.46:0/3386903643 conn(0x7ffa240ff570 msgr2=0x7ffa24108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 shutdown_connections 2024-04-17T07:59:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.312+0000 7ffa23200640 1 -- 172.21.15.46:0/3386903643 wait complete. 2024-04-17T07:59:21.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.554+0000 7fd743e00640 1 -- 172.21.15.46:0/3002297732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441051d0 msgr2=0x7fd744105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.554+0000 7fd743e00640 1 --2- 172.21.15.46:0/3002297732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441051d0 0x7fd744105650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd73002bc50 comp rx=0 tx=0).stop 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.554+0000 7fd743e00640 1 -- 172.21.15.46:0/3002297732 shutdown_connections 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.554+0000 7fd743e00640 1 --2- 172.21.15.46:0/3002297732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd744105b90 0x7fd74410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.554+0000 7fd743e00640 1 --2- 172.21.15.46:0/3002297732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7441051d0 0x7fd744105650 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.554+0000 7fd743e00640 1 --2- 172.21.15.46:0/3002297732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd744103fe0 0x7fd744104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.554+0000 7fd743e00640 1 -- 172.21.15.46:0/3002297732 >> 172.21.15.46:0/3002297732 conn(0x7fd7440ff570 msgr2=0x7fd744101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.555+0000 7fd743e00640 1 -- 172.21.15.46:0/3002297732 shutdown_connections 2024-04-17T07:59:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.555+0000 7fd743e00640 1 -- 172.21.15.46:0/3002297732 wait complete. 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 Processor -- start 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 -- start start 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd744103fe0 0x7fd74419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7441051d0 0x7fd74419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd744105b90 0x7fd7441a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410fc70 con 0x7fd744105b90 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410faf0 con 0x7fd7441051d0 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.556+0000 7fd743e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410fdf0 con 0x7fd744103fe0 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd743400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd744105b90 0x7fd7441a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7441051d0 0x7fd74419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd744103fe0 0x7fd74419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7441051d0 0x7fd74419a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32910/0 (socket says 172.21.15.46:32910) 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 -- 172.21.15.46:0/146962925 learned_addr learned my addr 172.21.15.46:0/146962925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 -- 172.21.15.46:0/146962925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd744103fe0 msgr2=0x7fd74419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd744103fe0 0x7fd74419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 -- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd744105b90 msgr2=0x7fd7441a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd744105b90 0x7fd7441a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 -- 172.21.15.46:0/146962925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441a1fc0 con 0x7fd7441051d0 2024-04-17T07:59:21.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742a00640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd744103fe0 0x7fd74419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.557+0000 7fd742000640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7441051d0 0x7fd74419a8e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd730009230 tx=0x7fd730005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.558+0000 7fd73b400640 1 -- 172.21.15.46:0/146962925 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730042020 con 0x7fd7441051d0 2024-04-17T07:59:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.558+0000 7fd73b400640 1 -- 172.21.15.46:0/146962925 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7300044d0 con 0x7fd7441051d0 2024-04-17T07:59:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.558+0000 7fd73b400640 1 -- 172.21.15.46:0/146962925 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73003e460 con 0x7fd7441051d0 2024-04-17T07:59:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.558+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7441a2250 con 0x7fd7441051d0 2024-04-17T07:59:21.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.558+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441a27f0 con 0x7fd7441051d0 2024-04-17T07:59:21.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.562+0000 7fd73b400640 1 -- 172.21.15.46:0/146962925 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd730045020 con 0x7fd7441051d0 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.563+0000 7fd74c20d740 1 Processor -- start 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.563+0000 7fd73b400640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7280a3c30 0x7fd7280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.563+0000 7fd74c20d740 1 -- start start 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.563+0000 7fd73b400640 1 -- 172.21.15.46:0/146962925 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd7300f2760 con 0x7fd7441051d0 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.563+0000 7fd743400640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7280a3c30 0x7fd7280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500c8160 0x5584500c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500cdcc0 0x5584500d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd74c20d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55845002e020 con 0x5584500c8b50 2024-04-17T07:59:21.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd74c20d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55844ffd4460 con 0x5584500cdcc0 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd74c20d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558450029fe0 con 0x5584500c8160 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd742a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500c8160 0x5584500c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd743400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500cdcc0 0x5584500d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd742a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500c8160 0x5584500c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50690/0 (socket says 172.21.15.46:50690) 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd742000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd742a00640 1 -- 172.21.15.46:0/2537980774 learned_addr learned my addr 172.21.15.46:0/2537980774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.564+0000 7fd743400640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7280a3c30 0x7fd7280a60f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005fd0 tx=0x7fd73c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd743400640 1 -- 172.21.15.46:0/2537980774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500c8160 msgr2=0x5584500c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd743400640 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500c8160 0x5584500c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd743400640 1 -- 172.21.15.46:0/2537980774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 msgr2=0x5584500cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd743400640 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd743400640 1 -- 172.21.15.46:0/2537980774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844ffdc040 con 0x5584500cdcc0 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd742a00640 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500c8160 0x5584500c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:21.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd742000640 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:21.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd743400640 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500cdcc0 0x5584500d0150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd73c004410 tx=0x7fd73c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52dcab735f87057b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd72f400640 1 -- 172.21.15.46:0/2537980774 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c026670 con 0x5584500cdcc0 2024-04-17T07:59:21.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd72f400640 1 -- 172.21.15.46:0/2537980774 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73c021070 con 0x5584500cdcc0 2024-04-17T07:59:21.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.565+0000 7fd72f400640 1 -- 172.21.15.46:0/2537980774 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0106a0 con 0x5584500cdcc0 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 -- 172.21.15.46:0/2537980774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500cdcc0 msgr2=0x5584500d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500cdcc0 0x5584500d0150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd73c004410 tx=0x7fd73c01b000 comp rx=0 tx=0).stop 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 -- 172.21.15.46:0/2537980774 shutdown_connections 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500cdcc0 0x5584500d0150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 --2- 172.21.15.46:0/2537980774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500c8160 0x5584500c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 -- 172.21.15.46:0/2537980774 >> 172.21.15.46:0/2537980774 conn(0x5584500c36d0 msgr2=0x5584500c3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 -- 172.21.15.46:0/2537980774 shutdown_connections 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.566+0000 7fd74c20d740 1 -- 172.21.15.46:0/2537980774 wait complete. 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.567+0000 7fd74c20d740 1 -- start start 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.567+0000 7fd74c20d740 1 Processor -- start 2024-04-17T07:59:21.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.567+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.567+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd74c20d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5584500d0c70 con 0x5584500c8160 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd74c20d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55844ffe7830 con 0x5584500c8b50 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd74c20d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5584500d0df0 con 0x5584500cdcc0 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd74c20d740 10 client.0 _ll_register_callbacks cb 0x55844ffb5fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40846/0 (socket says 172.21.15.46:40846) 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 -- 172.21.15.46:0/4095286950 learned_addr learned my addr 172.21.15.46:0/4095286950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd743400640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742000640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 -- 172.21.15.46:0/4095286950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 msgr2=0x5584500d9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 -- 172.21.15.46:0/4095286950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 msgr2=0x5584500d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742a00640 1 -- 172.21.15.46:0/4095286950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844ffeb920 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.568+0000 7fd742000640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd742a00640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d26d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd734012b10 tx=0x7fd73400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f6212cac21b151 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd723e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd723e00640 1 -- 172.21.15.46:0/4095286950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400fae0 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd723e00640 1 -- 172.21.15.46:0/4095286950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd73400fc80 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73c021070 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd723e00640 1 -- 172.21.15.46:0/4095286950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd734025050 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd74c20d740 10 client.15501 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.569+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55844ffb97f0 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd723e00640 1 -- 172.21.15.46:0/4095286950 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73c021070 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd723e00640 1 -- 172.21.15.46:0/4095286950 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fd734003cb0 con 0x5584500c8160 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd74c20d740 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:21.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd74c20d740 10 client.15501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T07:59:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd74c20d740 20 client.15501 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd74c20d740 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5584500de230 0x5584500e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd74c20d740 4 client.15501 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55845003ba40 con 0x5584500de230 2024-04-17T07:59:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.570+0000 7fd742000640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5584500de230 0x5584500e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.571+0000 7fd742000640 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5584500de230 0x5584500e06a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19f3129f62cb5ce0 server_cookie=a412e5cb991044d0 in_seq=0 out_seq=0 2024-04-17T07:59:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.571+0000 7fd723e00640 10 client.15501 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.571+0000 7fd723e00640 1 -- 172.21.15.46:0/4095286950 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55845003ba40 con 0x5584500de230 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.571+0000 7fd723e00640 10 client.15501 handle_command_reply: tid=0 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 client.15501 shutdown 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 msgr2=0x5584500d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d26d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd734012b10 tx=0x7fd73400f3c0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 shutdown_connections 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5584500de230 0x5584500e06a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 --2- 172.21.15.46:0/4095286950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d26d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 >> 172.21.15.46:0/4095286950 conn(0x5584500c36d0 msgr2=0x5584500c2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd723e00640 0 client.15501 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 shutdown_connections 2024-04-17T07:59:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 1 -- 172.21.15.46:0/4095286950 wait complete. 2024-04-17T07:59:21.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.572+0000 7fd74c20d740 20 client.15501 trim_cache size 0 max 16384 2024-04-17T07:59:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.587+0000 7fd74c20d740 1 Processor -- start 2024-04-17T07:59:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.587+0000 7fd74c20d740 1 -- start start 2024-04-17T07:59:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd74c20d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441af5b0 con 0x5584500c8160 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd74c20d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441ace70 con 0x5584500c8b50 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd74c20d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441aacc0 con 0x5584500cdcc0 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40862/0 (socket says 172.21.15.46:40862) 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742a00640 1 -- 172.21.15.46:0/3833473 learned_addr learned my addr 172.21.15.46:0/3833473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742000640 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd743400640 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742a00640 1 -- 172.21.15.46:0/3833473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 msgr2=0x5584500d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742a00640 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742a00640 1 -- 172.21.15.46:0/3833473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 msgr2=0x5584500d9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.588+0000 7fd742a00640 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.589+0000 7fd742a00640 1 -- 172.21.15.46:0/3833473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844fd403f0 con 0x5584500c8160 2024-04-17T07:59:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.589+0000 7fd742a00640 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d1cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd734009730 tx=0x7fd734010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a501d13fa09cb14d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.589+0000 7fd72ea00640 1 -- 172.21.15.46:0/3833473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400fae0 con 0x5584500c8160 2024-04-17T07:59:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.589+0000 7fd72ea00640 1 -- 172.21.15.46:0/3833473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73400fc80 con 0x5584500c8160 2024-04-17T07:59:21.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.589+0000 7fd72ea00640 1 -- 172.21.15.46:0/3833473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734005750 con 0x5584500c8160 2024-04-17T07:59:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 -- 172.21.15.46:0/3833473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 msgr2=0x5584500d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d1cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd734009730 tx=0x7fd734010900 comp rx=0 tx=0).stop 2024-04-17T07:59:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 -- 172.21.15.46:0/3833473 shutdown_connections 2024-04-17T07:59:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8b50 0x5584500d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3833473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8160 0x5584500d1cf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 -- 172.21.15.46:0/3833473 >> 172.21.15.46:0/3833473 conn(0x5584500c36d0 msgr2=0x5584500c71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 -- 172.21.15.46:0/3833473 shutdown_connections 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 -- 172.21.15.46:0/3833473 wait complete. 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.590+0000 7fd74c20d740 1 -- start start 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 1 Processor -- start 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8160 0x5584500d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441ab330 con 0x5584500c8b50 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441ac7a0 con 0x5584500c8160 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441aba00 con 0x5584500cdcc0 2024-04-17T07:59:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd74c20d740 10 client.0 _ll_register_callbacks cb 0x7fd734012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd742a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8160 0x5584500d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd742000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd743400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd742a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8160 0x5584500d7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:32942/0 (socket says 172.21.15.46:32942) 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.591+0000 7fd742a00640 1 -- 172.21.15.46:0/3486011750 learned_addr learned my addr 172.21.15.46:0/3486011750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd742a00640 1 -- 172.21.15.46:0/3486011750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 msgr2=0x5584500d8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd742a00640 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd742a00640 1 -- 172.21.15.46:0/3486011750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 msgr2=0x5584500d7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd742a00640 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500d7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd742a00640 1 -- 172.21.15.46:0/3486011750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844fe0a7d0 con 0x5584500c8160 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd742a00640 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8160 0x5584500d7040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd7340105a0 tx=0x7fd73400fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ca1fb99ca9cf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd723e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd723e00640 1 -- 172.21.15.46:0/3486011750 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734035020 con 0x5584500c8160 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73400fc80 con 0x5584500c8160 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.592+0000 7fd723e00640 1 -- 172.21.15.46:0/3486011750 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd734005110 con 0x5584500c8160 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd723e00640 1 -- 172.21.15.46:0/3486011750 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd734040050 con 0x5584500c8160 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 10 client.25640 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5584500d9c60 con 0x5584500c8160 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd723e00640 1 -- 172.21.15.46:0/3486011750 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73400fc80 con 0x5584500c8160 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd723e00640 1 -- 172.21.15.46:0/3486011750 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fd734003cc0 con 0x5584500c8160 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 10 client.25640 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 10 client.25640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 20 client.25640 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x558450144c60 0x558450147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 4 client.25640 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.593+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584500358b0 con 0x558450144c60 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.594+0000 7fd742000640 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x558450144c60 0x558450147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.594+0000 7fd742000640 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x558450144c60 0x558450147080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a89dd6c582935cc server_cookie=d9ba53cd48c1d1ba in_seq=0 out_seq=0 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.594+0000 7fd723e00640 10 client.25640 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd723e00640 1 -- 172.21.15.46:0/3486011750 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584500358b0 con 0x558450144c60 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd723e00640 10 client.25640 handle_command_reply: tid=0 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 client.25640 shutdown 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8160 msgr2=0x5584500d7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8160 0x5584500d7040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd7340105a0 tx=0x7fd73400fb50 comp rx=0 tx=0).stop 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 shutdown_connections 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584500cdcc0 0x5584500d8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x558450144c60 0x558450147080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5584500c8b50 0x5584500d7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 --2- 172.21.15.46:0/3486011750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584500c8160 0x5584500d7040 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd723e00640 0 client.25640 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 >> 172.21.15.46:0/3486011750 conn(0x5584500c36d0 msgr2=0x5584500c20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 shutdown_connections 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 1 -- 172.21.15.46:0/3486011750 wait complete. 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.595+0000 7fd74c20d740 20 client.25640 trim_cache size 0 max 16384 2024-04-17T07:59:21.602 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T07:59:21.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7280a3c30 msgr2=0x7fd7280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7280a3c30 0x7fd7280a60f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005fd0 tx=0x7fd73c0096a0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7441051d0 msgr2=0x7fd74419a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7441051d0 0x7fd74419a8e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd730009230 tx=0x7fd730005d50 comp rx=0 tx=0).stop 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 shutdown_connections 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd7280a3c30 0x7fd7280a60f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd744105b90 0x7fd7441a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7441051d0 0x7fd74419a8e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 --2- 172.21.15.46:0/146962925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd744103fe0 0x7fd74419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 >> 172.21.15.46:0/146962925 conn(0x7fd7440ff570 msgr2=0x7fd744108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 shutdown_connections 2024-04-17T07:59:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.596+0000 7fd743e00640 1 -- 172.21.15.46:0/146962925 wait complete. 2024-04-17T07:59:21.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:59:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 -- 172.21.15.46:0/2763345728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81085c0 msgr2=0x7faac81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 --2- 172.21.15.46:0/2763345728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81085c0 0x7faac81089e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faab8009be0 tx=0x7faab802bef0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 -- 172.21.15.46:0/2763345728 shutdown_connections 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 --2- 172.21.15.46:0/2763345728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac810a170 0x7faac8110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 --2- 172.21.15.46:0/2763345728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac81097b0 0x7faac8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 --2- 172.21.15.46:0/2763345728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81085c0 0x7faac81089e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 -- 172.21.15.46:0/2763345728 >> 172.21.15.46:0/2763345728 conn(0x7faac8077570 msgr2=0x7faac8079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 -- 172.21.15.46:0/2763345728 shutdown_connections 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.840+0000 7faacec00640 1 -- 172.21.15.46:0/2763345728 wait complete. 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.841+0000 7faacec00640 1 Processor -- start 2024-04-17T07:59:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faacec00640 1 -- start start 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faacec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac81085c0 0x7faac819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faacec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81097b0 0x7faac819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faacec00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac810a170 0x7faac81a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faacec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faac8114230 con 0x7faac81085c0 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faacec00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faac81140b0 con 0x7faac810a170 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faacec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faac81143b0 con 0x7faac81097b0 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faac7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac81085c0 0x7faac819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faac7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81097b0 0x7faac819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faac7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac81085c0 0x7faac819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40888/0 (socket says 172.21.15.46:40888) 2024-04-17T07:59:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.842+0000 7faac7e00640 1 -- 172.21.15.46:0/1257685470 learned_addr learned my addr 172.21.15.46:0/1257685470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.843+0000 7faaccc00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac810a170 0x7faac81a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.843+0000 7faac7e00640 1 -- 172.21.15.46:0/1257685470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81097b0 msgr2=0x7faac819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.843+0000 7faac7e00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81097b0 0x7faac819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.843+0000 7faac7e00640 1 -- 172.21.15.46:0/1257685470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac810a170 msgr2=0x7faac81a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.843+0000 7faac7e00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac810a170 0x7faac81a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.843+0000 7faac7e00640 1 -- 172.21.15.46:0/1257685470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac81a6520 con 0x7faac81085c0 2024-04-17T07:59:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.843+0000 7faac7e00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac81085c0 0x7faac819e900 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faab8036040 tx=0x7faab8033580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.844+0000 7faac4c00640 1 -- 172.21.15.46:0/1257685470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8043020 con 0x7faac81085c0 2024-04-17T07:59:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.844+0000 7faac4c00640 1 -- 172.21.15.46:0/1257685470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab8003e50 con 0x7faac81085c0 2024-04-17T07:59:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.844+0000 7faac4c00640 1 -- 172.21.15.46:0/1257685470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8034690 con 0x7faac81085c0 2024-04-17T07:59:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.844+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac81a67b0 con 0x7faac81085c0 2024-04-17T07:59:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.844+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac81a6d00 con 0x7faac81085c0 2024-04-17T07:59:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.847+0000 7faac4c00640 1 -- 172.21.15.46:0/1257685470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faab8046020 con 0x7faac81085c0 2024-04-17T07:59:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.847+0000 7faad19bd740 1 Processor -- start 2024-04-17T07:59:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.847+0000 7faad19bd740 1 -- start start 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faad19bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faad19bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a6fb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faad19bd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faad19bd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce9a65c020 con 0x55ce9a6fbb00 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faad19bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ce9a602460 con 0x55ce9a6f6990 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faad19bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ce9a657fe0 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faac7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50732/0 (socket says 172.21.15.46:50732) 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 -- 172.21.15.46:0/1202812215 learned_addr learned my addr 172.21.15.46:0/1202812215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faac7e00640 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a6fb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 -- 172.21.15.46:0/1202812215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 msgr2=0x55ce9a6fb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a6fb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 -- 172.21.15.46:0/1202812215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 msgr2=0x55ce9a6fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6fdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.848+0000 7faaccc00640 1 -- 172.21.15.46:0/1202812215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce9a60a040 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.849+0000 7faaccc00640 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6f63c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faac0009c60 tx=0x7faac0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc96c415aa096a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.849+0000 7faac4c00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faaac0a3bb0 0x7faaac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.849+0000 7faabd200640 1 -- 172.21.15.46:0/1202812215 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac001e070 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.849+0000 7faabd200640 1 -- 172.21.15.46:0/1202812215 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0019b10 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.849+0000 7faac4c00640 1 -- 172.21.15.46:0/1257685470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7faab80f2da0 con 0x7faac81085c0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.849+0000 7faabd200640 1 -- 172.21.15.46:0/1202812215 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac000ea70 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.849+0000 7faac7e00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faaac0a3bb0 0x7faaac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 -- 172.21.15.46:0/1202812215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 msgr2=0x55ce9a6f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faac7e00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faaac0a3bb0 0x7faaac0a6070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faab80f0880 tx=0x7faab80f6880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6f63c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faac0009c60 tx=0x7faac0011810 comp rx=0 tx=0).stop 2024-04-17T07:59:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 -- 172.21.15.46:0/1202812215 shutdown_connections 2024-04-17T07:59:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a6fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 --2- 172.21.15.46:0/1202812215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6f63c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 -- 172.21.15.46:0/1202812215 >> 172.21.15.46:0/1202812215 conn(0x55ce9a6f1510 msgr2=0x55ce9a6f1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 -- 172.21.15.46:0/1202812215 shutdown_connections 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.850+0000 7faad19bd740 1 -- 172.21.15.46:0/1202812215 wait complete. 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 -- start start 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 Processor -- start 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a700520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a700a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a707aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce9a6feab0 con 0x55ce9a6fbb00 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ce9a615830 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ce9a6fec30 con 0x55ce9a6f6990 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.851+0000 7faad19bd740 10 client.0 _ll_register_callbacks cb 0x55ce9a5e3fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a700a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a700a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50734/0 (socket says 172.21.15.46:50734) 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faaccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a707aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 -- 172.21.15.46:0/1501398926 learned_addr learned my addr 172.21.15.46:0/1501398926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faaccc00640 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a700520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 -- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f5fa0 msgr2=0x55ce9a700520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a700520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 -- 172.21.15.46:0/1501398926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 msgr2=0x55ce9a707aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a707aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 -- 172.21.15.46:0/1501398926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce9a619920 con 0x55ce9a6f6990 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faaccc00640 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a700520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faac7400640 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a700a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faab40126e0 tx=0x7faab4012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a78ade9681c9d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.852+0000 7faab1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.853+0000 7faab1600640 1 -- 172.21.15.46:0/1501398926 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab400f8c0 con 0x55ce9a6f6990 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.853+0000 7faab1600640 1 -- 172.21.15.46:0/1501398926 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7faab400fa60 con 0x55ce9a6f6990 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.853+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac0019b10 con 0x55ce9a6f6990 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.853+0000 7faab1600640 1 -- 172.21.15.46:0/1501398926 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faab400fc60 con 0x55ce9a6f6990 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.853+0000 7faad19bd740 10 client.25636 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.853+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ce9a5e77f0 con 0x55ce9a6f6990 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.853+0000 7faab1600640 1 -- 172.21.15.46:0/1501398926 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0019b10 con 0x55ce9a6f6990 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faab1600640 1 -- 172.21.15.46:0/1501398926 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7faab401c490 con 0x55ce9a6f6990 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faad19bd740 10 client.25636 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faad19bd740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faad19bd740 20 client.25636 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faad19bd740 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a70c140 0x55ce9a70e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faad19bd740 4 client.25636 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce9a669a40 con 0x55ce9a70c140 2024-04-17T07:59:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.854+0000 7faaccc00640 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a70c140 0x55ce9a70e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.855+0000 7faaccc00640 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a70c140 0x55ce9a70e580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb84a43474621865 server_cookie=c7c4f6f3095118a5 in_seq=0 out_seq=0 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.855+0000 7faab1600640 10 client.25636 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faab1600640 1 -- 172.21.15.46:0/1501398926 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce9a669a40 con 0x55ce9a70c140 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faab1600640 10 client.25636 handle_command_reply: tid=0 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 client.25636 shutdown 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 msgr2=0x55ce9a700a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a700a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faab40126e0 tx=0x7faab4012ba0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 shutdown_connections 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a70c140 0x55ce9a70e580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a707aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a700a60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faab1600640 0 client.25636 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 --2- 172.21.15.46:0/1501398926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a700520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 >> 172.21.15.46:0/1501398926 conn(0x55ce9a6f1510 msgr2=0x55ce9a6f0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 shutdown_connections 2024-04-17T07:59:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 1 -- 172.21.15.46:0/1501398926 wait complete. 2024-04-17T07:59:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.856+0000 7faad19bd740 20 client.25636 trim_cache size 0 max 16384 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 Processor -- start 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 -- start start 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6efe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a707a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faac81b3ae0 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faac81b13a0 con 0x55ce9a6fbb00 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faad19bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faac81af1f0 con 0x55ce9a6f6990 2024-04-17T07:59:21.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faaccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faaccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6efe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40908/0 (socket says 172.21.15.46:40908) 2024-04-17T07:59:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.872+0000 7faaccc00640 1 -- 172.21.15.46:0/1529644466 learned_addr learned my addr 172.21.15.46:0/1529644466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faaccc00640 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faac7400640 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a707a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faaccc00640 1 -- 172.21.15.46:0/1529644466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 msgr2=0x55ce9a707a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faaccc00640 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a707a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faaccc00640 1 -- 172.21.15.46:0/1529644466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6fbb00 msgr2=0x55ce9a6f03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faaccc00640 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6f03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faaccc00640 1 -- 172.21.15.46:0/1529644466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce9a36e3f0 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faaccc00640 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6efe80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faac001c040 tx=0x7faac0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db25878e0d3a4a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faab3e00640 1 -- 172.21.15.46:0/1529644466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac001e040 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faab3e00640 1 -- 172.21.15.46:0/1529644466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0005520 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.873+0000 7faab3e00640 1 -- 172.21.15.46:0/1529644466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0004ac0 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 -- 172.21.15.46:0/1529644466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6f5fa0 msgr2=0x55ce9a6efe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6efe80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faac001c040 tx=0x7faac0009260 comp rx=0 tx=0).stop 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 -- 172.21.15.46:0/1529644466 shutdown_connections 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6fbb00 0x55ce9a6f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f6990 0x55ce9a707a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 --2- 172.21.15.46:0/1529644466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a6efe80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 -- 172.21.15.46:0/1529644466 >> 172.21.15.46:0/1529644466 conn(0x55ce9a6f1510 msgr2=0x55ce9a6f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 -- 172.21.15.46:0/1529644466 shutdown_connections 2024-04-17T07:59:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.874+0000 7faad19bd740 1 -- 172.21.15.46:0/1529644466 wait complete. 2024-04-17T07:59:21.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 -- start start 2024-04-17T07:59:21.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 Processor -- start 2024-04-17T07:59:21.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a704f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a7054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a706420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faac81af860 con 0x55ce9a6fbb00 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faac81b0cd0 con 0x55ce9a6f6990 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faac81aff30 con 0x55ce9a6f5fa0 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.875+0000 7faad19bd740 10 client.0 _ll_register_callbacks cb 0x7faab4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faaccc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a704f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faaccc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a704f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50762/0 (socket says 172.21.15.46:50762) 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faaccc00640 1 -- 172.21.15.46:0/1616862697 learned_addr learned my addr 172.21.15.46:0/1616862697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faac7400640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a7054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faaccc00640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a706420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faac7400640 1 -- 172.21.15.46:0/1616862697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 msgr2=0x55ce9a704f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faac7400640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a704f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faac7400640 1 -- 172.21.15.46:0/1616862697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 msgr2=0x55ce9a706420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faac7400640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a706420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faac7400640 1 -- 172.21.15.46:0/1616862697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce9a4387d0 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faaccc00640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a706420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faac7400640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a7054d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ce9a6f1c30 tx=0x7faab400f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d6c4162e6c2fca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faab1600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.876+0000 7faab1600640 1 -- 172.21.15.46:0/1616862697 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4030060 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.877+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac0005520 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.877+0000 7faab1600640 1 -- 172.21.15.46:0/1616862697 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7faab401c420 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.877+0000 7faab1600640 1 -- 172.21.15.46:0/1616862697 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faab401c640 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.877+0000 7faad19bd740 10 client.25658 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.877+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ce9a707bb0 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.877+0000 7faab1600640 1 -- 172.21.15.46:0/1616862697 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0005520 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faab1600640 1 -- 172.21.15.46:0/1616862697 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7faab4003a10 con 0x55ce9a6f6990 2024-04-17T07:59:21.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faad19bd740 10 client.25658 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faad19bd740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faad19bd740 20 client.25658 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faad19bd740 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a772bd0 0x55ce9a774ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faad19bd740 4 client.25658 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce9a6638b0 con 0x55ce9a772bd0 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faaccc00640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a772bd0 0x55ce9a774ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.878+0000 7faaccc00640 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a772bd0 0x55ce9a774ff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c330ad2454c3d85 server_cookie=ce76eceb09a02133 in_seq=0 out_seq=0 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faab1600640 10 client.25658 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faab1600640 1 -- 172.21.15.46:0/1616862697 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce9a6638b0 con 0x55ce9a772bd0 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faab1600640 10 client.25658 handle_command_reply: tid=0 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 client.25658 shutdown 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 msgr2=0x55ce9a7054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a7054d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ce9a6f1c30 tx=0x7faab400f810 comp rx=0 tx=0).stop 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 shutdown_connections 2024-04-17T07:59:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ce9a772bd0 0x55ce9a774ff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9a6fbb00 0x55ce9a706420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ce9a6f6990 0x55ce9a7054d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faab1600640 0 client.25658 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 --2- 172.21.15.46:0/1616862697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ce9a6f5fa0 0x55ce9a704f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 >> 172.21.15.46:0/1616862697 conn(0x55ce9a6f1510 msgr2=0x55ce9a6f4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 shutdown_connections 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 1 -- 172.21.15.46:0/1616862697 wait complete. 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.879+0000 7faad19bd740 20 client.25658 trim_cache size 0 max 16384 2024-04-17T07:59:21.882 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faaac0a3bb0 msgr2=0x7faaac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faaac0a3bb0 0x7faaac0a6070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faab80f0880 tx=0x7faab80f6880 comp rx=0 tx=0).stop 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac81085c0 msgr2=0x7faac819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac81085c0 0x7faac819e900 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faab8036040 tx=0x7faab8033580 comp rx=0 tx=0).stop 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 shutdown_connections 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7faaac0a3bb0 0x7faaac0a6070 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac810a170 0x7faac81a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faac81097b0 0x7faac819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 --2- 172.21.15.46:0/1257685470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac81085c0 0x7faac819e900 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 >> 172.21.15.46:0/1257685470 conn(0x7faac8077570 msgr2=0x7faac810cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 shutdown_connections 2024-04-17T07:59:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:21.880+0000 7faacec00640 1 -- 172.21.15.46:0/1257685470 wait complete. 2024-04-17T07:59:21.894 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:21 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/315037148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:59:22.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:21 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/315037148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:59:22.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:21 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/315037148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:59:22.898 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:59:22.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:59:22.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:59:22.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:22 smithi046 ceph-mon[32038]: pgmap v443: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-17T07:59:23.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:22 smithi062 ceph-mon[36507]: pgmap v443: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-17T07:59:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 -- 172.21.15.46:0/3776229351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 msgr2=0x7f174c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 --2- 172.21.15.46:0/3776229351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 0x7f174c101280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1738009860 tx=0x7f173802bb70 comp rx=0 tx=0).stop 2024-04-17T07:59:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 -- 172.21.15.46:0/3776229351 shutdown_connections 2024-04-17T07:59:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 --2- 172.21.15.46:0/3776229351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1017c0 0x7f174c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 --2- 172.21.15.46:0/3776229351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 0x7f174c101280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 --2- 172.21.15.46:0/3776229351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f174c1000a0 0x7f174c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 -- 172.21.15.46:0/3776229351 >> 172.21.15.46:0/3776229351 conn(0x7f174c0fb9f0 msgr2=0x7f174c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 -- 172.21.15.46:0/3776229351 shutdown_connections 2024-04-17T07:59:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.133+0000 7f1752c00640 1 -- 172.21.15.46:0/3776229351 wait complete. 2024-04-17T07:59:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.134+0000 7f1752c00640 1 Processor -- start 2024-04-17T07:59:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752c00640 1 -- start start 2024-04-17T07:59:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1000a0 0x7f174c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 0x7f174c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f174c1017c0 0x7f174c1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10fcb0 con 0x7f174c1000a0 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10fb30 con 0x7f174c1017c0 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10fe30 con 0x7f174c100e00 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1751800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1000a0 0x7f174c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1750e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 0x7f174c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1751800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1000a0 0x7f174c19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40928/0 (socket says 172.21.15.46:40928) 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1751800640 1 -- 172.21.15.46:0/381994942 learned_addr learned my addr 172.21.15.46:0/381994942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.135+0000 7f1752200640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f174c1017c0 0x7f174c1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1751800640 1 -- 172.21.15.46:0/381994942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 msgr2=0x7f174c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1751800640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 0x7f174c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1751800640 1 -- 172.21.15.46:0/381994942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f174c1017c0 msgr2=0x7f174c1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1751800640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f174c1017c0 0x7f174c1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1751800640 1 -- 172.21.15.46:0/381994942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c1a1f80 con 0x7f174c1000a0 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1752200640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f174c1017c0 0x7f174c1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1751800640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1000a0 0x7f174c19a360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f174000e9f0 tx=0x7f174000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f174a000640 1 -- 172.21.15.46:0/381994942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174000cd80 con 0x7f174c1000a0 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.136+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c1a2270 con 0x7f174c1000a0 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.137+0000 7f174a000640 1 -- 172.21.15.46:0/381994942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1740010910 con 0x7f174c1000a0 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.137+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c1a27e0 con 0x7f174c1000a0 2024-04-17T07:59:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.137+0000 7f174a000640 1 -- 172.21.15.46:0/381994942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740026060 con 0x7f174c1000a0 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 Processor -- start 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 -- start start 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f174a000640 1 -- 172.21.15.46:0/381994942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1740010ac0 con 0x7f174c1000a0 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92ce64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce6a90 0x55ab92ceb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92cebc00 0x55ab92cee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ab92c4bcb0 con 0x55ab92ce60a0 2024-04-17T07:59:23.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ab92bf2370 con 0x55ab92ce6a90 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.140+0000 7f1757024740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ab92c47c70 con 0x55ab92cebc00 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92ce64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92ce64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40932/0 (socket says 172.21.15.46:40932) 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 -- 172.21.15.46:0/2039411952 learned_addr learned my addr 172.21.15.46:0/2039411952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1750e00640 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92cebc00 0x55ab92cee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1751800640 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce6a90 0x55ab92ceb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 -- 172.21.15.46:0/2039411952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92cebc00 msgr2=0x55ab92cee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92cebc00 0x55ab92cee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 -- 172.21.15.46:0/2039411952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce6a90 msgr2=0x55ab92ceb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce6a90 0x55ab92ceb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1752200640 1 -- 172.21.15.46:0/2039411952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab92c11910 con 0x55ab92ce60a0 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1751800640 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce6a90 0x55ab92ceb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.141+0000 7f1750e00640 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92cebc00 0x55ab92cee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.142+0000 7f1752200640 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92ce64c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1744009fd0 tx=0x7f174400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b98402a02d8ba28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.142+0000 7f173ea00640 1 -- 172.21.15.46:0/2039411952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174401d070 con 0x55ab92ce60a0 2024-04-17T07:59:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.142+0000 7f173ea00640 1 -- 172.21.15.46:0/2039411952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1744003e50 con 0x55ab92ce60a0 2024-04-17T07:59:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.142+0000 7f173ea00640 1 -- 172.21.15.46:0/2039411952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17440183f0 con 0x55ab92ce60a0 2024-04-17T07:59:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.142+0000 7f174a000640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17340a3bb0 0x7f17340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.142+0000 7f174a000640 1 -- 172.21.15.46:0/381994942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f17400d0bc0 con 0x7f174c1000a0 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.142+0000 7f1751800640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17340a3bb0 0x7f17340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 -- 172.21.15.46:0/2039411952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 msgr2=0x55ab92ce64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92ce64c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1744009fd0 tx=0x7f174400ec80 comp rx=0 tx=0).stop 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 -- 172.21.15.46:0/2039411952 shutdown_connections 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92cebc00 0x55ab92cee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce6a90 0x55ab92ceb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 --2- 172.21.15.46:0/2039411952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92ce64c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 -- 172.21.15.46:0/2039411952 >> 172.21.15.46:0/2039411952 conn(0x55ab92ce1610 msgr2=0x55ab92ce1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 -- 172.21.15.46:0/2039411952 shutdown_connections 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1751800640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17340a3bb0 0x7f17340a6070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f174009caf0 tx=0x7f174009c6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 -- 172.21.15.46:0/2039411952 wait complete. 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.143+0000 7f1757024740 1 -- start start 2024-04-17T07:59:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 1 Processor -- start 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92cebc00 0x55ab92cf7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ab92ceebb0 con 0x55ab92cebc00 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ab92c05730 con 0x55ab92ce60a0 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ab92ceed30 con 0x55ab92ce6a90 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1757024740 10 client.0 _ll_register_callbacks cb 0x55ab92c07b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.144+0000 7f1752200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92cebc00 0x55ab92cf7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1752200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50804/0 (socket says 172.21.15.46:50804) 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 -- 172.21.15.46:0/3844262855 learned_addr learned my addr 172.21.15.46:0/3844262855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1752200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33032/0 (socket says 172.21.15.46:33032) 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 -- 172.21.15.46:0/3844262855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 msgr2=0x55ab92cf05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 -- 172.21.15.46:0/3844262855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92cebc00 msgr2=0x55ab92cf7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92cebc00 0x55ab92cf7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 -- 172.21.15.46:0/3844262855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab92a267e0 con 0x55ab92ce6a90 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1752200640 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92cebc00 0x55ab92cf7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1750e00640 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf0b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f173802c180 tx=0x7f1738009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd671b9682f7c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.145+0000 7f1733400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.146+0000 7f1733400640 1 -- 172.21.15.46:0/3844262855 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738043070 con 0x55ab92ce6a90 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.146+0000 7f1733400640 1 -- 172.21.15.46:0/3844262855 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f1738004040 con 0x55ab92ce6a90 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.146+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1744003e50 con 0x55ab92ce6a90 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.146+0000 7f1733400640 1 -- 172.21.15.46:0/3844262855 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f173804f050 con 0x55ab92ce6a90 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.146+0000 7f1757024740 10 client.25654 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.146+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ab92a86f70 con 0x55ab92ce6a90 2024-04-17T07:59:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.146+0000 7f1733400640 1 -- 172.21.15.46:0/3844262855 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1744003e50 con 0x55ab92ce6a90 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1733400640 1 -- 172.21.15.46:0/3844262855 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f1738032640 con 0x55ab92ce6a90 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1757024740 10 client.25654 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1757024740 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1757024740 20 client.25654 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1757024740 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cfc430 0x55ab92cfe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1757024740 4 client.25654 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab92c596d0 con 0x55ab92cfc430 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.147+0000 7f1752200640 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cfc430 0x55ab92cfe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.148+0000 7f1752200640 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cfc430 0x55ab92cfe850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550ece7e29f38844 server_cookie=762b94a575ab69cc in_seq=0 out_seq=0 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.148+0000 7f1733400640 10 client.25654 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1733400640 1 -- 172.21.15.46:0/3844262855 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab92c596d0 con 0x55ab92cfc430 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1733400640 10 client.25654 handle_command_reply: tid=0 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 client.25654 shutdown 2024-04-17T07:59:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 msgr2=0x55ab92cf0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf0b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f173802c180 tx=0x7f1738009490 comp rx=0 tx=0).stop 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 shutdown_connections 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cfc430 0x55ab92cfe850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92cebc00 0x55ab92cf7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf0b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 --2- 172.21.15.46:0/3844262855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 >> 172.21.15.46:0/3844262855 conn(0x55ab92ce1610 msgr2=0x55ab92ce0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1733400640 0 client.25654 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 shutdown_connections 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 1 -- 172.21.15.46:0/3844262855 wait complete. 2024-04-17T07:59:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.149+0000 7f1757024740 20 client.25654 trim_cache size 0 max 16384 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 Processor -- start 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 -- start start 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce1610 0x55ab92ce4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92d03110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1af5a0 con 0x55ab92ce1610 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1ace60 con 0x55ab92ce60a0 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1757024740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1aacb0 con 0x55ab92ce6a90 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.169+0000 7f1752200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce1610 0x55ab92ce4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce1610 0x55ab92ce4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40942/0 (socket says 172.21.15.46:40942) 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 -- 172.21.15.46:0/139560993 learned_addr learned my addr 172.21.15.46:0/139560993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1750e00640 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92d03110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 -- 172.21.15.46:0/139560993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 msgr2=0x55ab92d03110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92d03110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 -- 172.21.15.46:0/139560993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 msgr2=0x55ab92cf7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 -- 172.21.15.46:0/139560993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9291e170 con 0x55ab92ce1610 2024-04-17T07:59:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1750e00640 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.170+0000 7f1752200640 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce1610 0x55ab92ce4f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1744009fd0 tx=0x7f17440095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb054af824e856e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f173e000640 1 -- 172.21.15.46:0/139560993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174401d040 con 0x55ab92ce1610 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f173e000640 1 -- 172.21.15.46:0/139560993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174401e040 con 0x55ab92ce1610 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f173e000640 1 -- 172.21.15.46:0/139560993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744004c60 con 0x55ab92ce1610 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 -- 172.21.15.46:0/139560993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce1610 msgr2=0x55ab92ce4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce1610 0x55ab92ce4f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1744009fd0 tx=0x7f17440095c0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 -- 172.21.15.46:0/139560993 shutdown_connections 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92d03110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce60a0 0x55ab92cf7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 --2- 172.21.15.46:0/139560993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce1610 0x55ab92ce4f80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 -- 172.21.15.46:0/139560993 >> 172.21.15.46:0/139560993 conn(0x55ab92ce1d60 msgr2=0x55ab92ce4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 -- 172.21.15.46:0/139560993 shutdown_connections 2024-04-17T07:59:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.171+0000 7f1757024740 1 -- 172.21.15.46:0/139560993 wait complete. 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 -- start start 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 Processor -- start 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce1610 0x55ab92cf57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92cf5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1ab320 con 0x55ab92ce60a0 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1ac790 con 0x55ab92ce1610 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.172+0000 7f1757024740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1ab9f0 con 0x55ab92ce6a90 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1757024740 10 client.0 _ll_register_callbacks cb 0x7f1740005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1750e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92cf5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce1610 0x55ab92cf57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1750e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92cf5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40950/0 (socket says 172.21.15.46:40950) 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1750e00640 1 -- 172.21.15.46:0/555671249 learned_addr learned my addr 172.21.15.46:0/555671249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 -- 172.21.15.46:0/555671249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 msgr2=0x55ab92cf6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 -- 172.21.15.46:0/555671249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 msgr2=0x55ab92cf5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92cf5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 -- 172.21.15.46:0/555671249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab92cf7300 con 0x55ab92ce1610 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1750e00640 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92cf5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1752200640 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce1610 0x55ab92cf57c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1744000f80 tx=0x7f1744005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d653365288ee02e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.173+0000 7f1733400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.174+0000 7f1733400640 1 -- 172.21.15.46:0/555671249 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174401d040 con 0x55ab92ce1610 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.174+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f174401e040 con 0x55ab92ce1610 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.174+0000 7f1733400640 1 -- 172.21.15.46:0/555671249 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f174400cce0 con 0x55ab92ce1610 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.174+0000 7f1733400640 1 -- 172.21.15.46:0/555671249 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1744003a10 con 0x55ab92ce1610 2024-04-17T07:59:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.174+0000 7f1757024740 10 client.25676 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.174+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ab92d62f40 con 0x55ab92ce1610 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.174+0000 7f1733400640 1 -- 172.21.15.46:0/555671249 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174401e040 con 0x55ab92ce1610 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1733400640 1 -- 172.21.15.46:0/555671249 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f1744023040 con 0x55ab92ce1610 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1757024740 10 client.25676 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1757024740 10 client.25676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1757024740 20 client.25676 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1757024740 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cebc00 0x55ab92d65650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1757024740 4 client.25676 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab92c53540 con 0x55ab92cebc00 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1750e00640 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cebc00 0x55ab92d65650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.175+0000 7f1750e00640 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cebc00 0x55ab92d65650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b734551abccc694 server_cookie=d03423c15d61c07d in_seq=0 out_seq=0 2024-04-17T07:59:23.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.176+0000 7f1733400640 10 client.25676 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1733400640 1 -- 172.21.15.46:0/555671249 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab92c53540 con 0x55ab92cebc00 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1733400640 10 client.25676 handle_command_reply: tid=0 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 client.25676 shutdown 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce1610 msgr2=0x55ab92cf57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce1610 0x55ab92cf57c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1744000f80 tx=0x7f1744005bd0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 shutdown_connections 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ab92cebc00 0x55ab92d65650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ab92ce6a90 0x55ab92cf6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab92ce60a0 0x55ab92cf5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 --2- 172.21.15.46:0/555671249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ab92ce1610 0x55ab92cf57c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1733400640 0 client.25676 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 >> 172.21.15.46:0/555671249 conn(0x55ab92ce1d60 msgr2=0x55ab92ce5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 shutdown_connections 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 1 -- 172.21.15.46:0/555671249 wait complete. 2024-04-17T07:59:23.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.177+0000 7f1757024740 20 client.25676 trim_cache size 0 max 16384 2024-04-17T07:59:23.181 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:59:23.181 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-17T07:59:23.181 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ab9ab8b1-fb93-40da-a0ea-f70b0a52dc8c", 2024-04-17T07:59:23.181 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-17T07:59:23.181 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17340a3bb0 msgr2=0x7f17340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17340a3bb0 0x7f17340a6070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f174009caf0 tx=0x7f174009c6f0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1000a0 msgr2=0x7f174c19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1000a0 0x7f174c19a360 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f174000e9f0 tx=0x7f174000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 shutdown_connections 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17340a3bb0 0x7f17340a6070 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f174c1017c0 0x7f174c1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f174c100e00 0x7f174c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 --2- 172.21.15.46:0/381994942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c1000a0 0x7f174c19a360 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 >> 172.21.15.46:0/381994942 conn(0x7f174c0fb9f0 msgr2=0x7f174c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 shutdown_connections 2024-04-17T07:59:23.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.178+0000 7f1752c00640 1 -- 172.21.15.46:0/381994942 wait complete. 2024-04-17T07:59:23.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:59:23.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:59:23.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:22 smithi053 ceph-mon[36437]: pgmap v443: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-17T07:59:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 -- 172.21.15.46:0/74883815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a8102fe0 msgr2=0x7f77a8103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 --2- 172.21.15.46:0/74883815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a8102fe0 0x7f77a8103460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7798009be0 tx=0x7f779802bef0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 -- 172.21.15.46:0/74883815 shutdown_connections 2024-04-17T07:59:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 --2- 172.21.15.46:0/74883815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a81039a0 0x7f77a810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 --2- 172.21.15.46:0/74883815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a8102fe0 0x7f77a8103460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 --2- 172.21.15.46:0/74883815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 0x7f77a8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 -- 172.21.15.46:0/74883815 >> 172.21.15.46:0/74883815 conn(0x7f77a80fd360 msgr2=0x7f77a80ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 -- 172.21.15.46:0/74883815 shutdown_connections 2024-04-17T07:59:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.433+0000 7f77a7200640 1 -- 172.21.15.46:0/74883815 wait complete. 2024-04-17T07:59:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 Processor -- start 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 -- start start 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 0x7f77a819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a8102fe0 0x7f77a819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a81039a0 0x7f77a81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77a810dac0 con 0x7f77a8101df0 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77a810d940 con 0x7f77a8102fe0 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.434+0000 7f77a7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77a810dc40 con 0x7f77a81039a0 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 0x7f77a819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 0x7f77a819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40958/0 (socket says 172.21.15.46:40958) 2024-04-17T07:59:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 -- 172.21.15.46:0/772708346 learned_addr learned my addr 172.21.15.46:0/772708346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a6800640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a81039a0 0x7f77a81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5400640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a8102fe0 0x7f77a819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 -- 172.21.15.46:0/772708346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a81039a0 msgr2=0x7f77a81a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a81039a0 0x7f77a81a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 -- 172.21.15.46:0/772708346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a8102fe0 msgr2=0x7f77a819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a8102fe0 0x7f77a819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 -- 172.21.15.46:0/772708346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a81a1f90 con 0x7f77a8101df0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a6800640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a81039a0 0x7f77a81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.435+0000 7f77a5e00640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 0x7f77a819a370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f779000e9f0 tx=0x7f779000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.436+0000 7f779ea00640 1 -- 172.21.15.46:0/772708346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779000ccc0 con 0x7f77a8101df0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.436+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a81a2280 con 0x7f77a8101df0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.436+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81a27f0 con 0x7f77a8101df0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.436+0000 7f779ea00640 1 -- 172.21.15.46:0/772708346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77900035f0 con 0x7f77a8101df0 2024-04-17T07:59:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.436+0000 7f779ea00640 1 -- 172.21.15.46:0/772708346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779000ce60 con 0x7f77a8101df0 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.439+0000 7f779ea00640 1 -- 172.21.15.46:0/772708346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7790004020 con 0x7f77a8101df0 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.439+0000 7f77af80c740 1 Processor -- start 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.439+0000 7f77af80c740 1 -- start start 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77af80c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b53902790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77af80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902d60 0x564b53907860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77af80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53907ed0 0x564b5390a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77af80c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564b53869120 con 0x564b53902370 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77af80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564b5380f340 con 0x564b53902d60 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77af80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564b538650e0 con 0x564b53907ed0 2024-04-17T07:59:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b53902790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b53902790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40970/0 (socket says 172.21.15.46:40970) 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77a6800640 1 -- 172.21.15.46:0/1924217430 learned_addr learned my addr 172.21.15.46:0/1924217430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902d60 0x564b53907860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.440+0000 7f77a5400640 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53907ed0 0x564b5390a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.441+0000 7f77a5400640 1 -- 172.21.15.46:0/1924217430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902d60 msgr2=0x564b53907860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.441+0000 7f77a5400640 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902d60 0x564b53907860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.441+0000 7f77a5400640 1 -- 172.21.15.46:0/1924217430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 msgr2=0x564b53902790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.441+0000 7f77a5400640 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b53902790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.441+0000 7f77a5400640 1 -- 172.21.15.46:0/1924217430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b536457d0 con 0x564b53907ed0 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.441+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902d60 0x564b53907860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.441+0000 7f77a5400640 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53907ed0 0x564b5390a360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f779802c500 tx=0x7f7798009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570f2d536e963489 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f7797000640 1 -- 172.21.15.46:0/1924217430 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798043030 con 0x564b53907ed0 2024-04-17T07:59:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f7797000640 1 -- 172.21.15.46:0/1924217430 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798033c80 con 0x564b53907ed0 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f7797000640 1 -- 172.21.15.46:0/1924217430 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779802c980 con 0x564b53907ed0 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f779ea00640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f778c0a3bb0 0x7f778c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f779ea00640 1 -- 172.21.15.46:0/772708346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f77900d0b20 con 0x7f77a8101df0 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f77a6800640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f778c0a3bb0 0x7f778c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f77af80c740 1 -- 172.21.15.46:0/1924217430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53907ed0 msgr2=0x564b5390a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.442+0000 7f77af80c740 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53907ed0 0x564b5390a360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f779802c500 tx=0x7f7798009300 comp rx=0 tx=0).stop 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 -- 172.21.15.46:0/1924217430 shutdown_connections 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53907ed0 0x564b5390a360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902d60 0x564b53907860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 --2- 172.21.15.46:0/1924217430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b53902790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 -- 172.21.15.46:0/1924217430 >> 172.21.15.46:0/1924217430 conn(0x564b538fd8e0 msgr2=0x564b538fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77a6800640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f778c0a3bb0 0x7f778c0a6070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x564b539035c0 tx=0x7f77a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 -- 172.21.15.46:0/1924217430 shutdown_connections 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 -- 172.21.15.46:0/1924217430 wait complete. 2024-04-17T07:59:23.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.443+0000 7f77af80c740 1 -- start start 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 1 Processor -- start 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b5390c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b5390cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53907ed0 0x564b53913d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564b5390ae20 con 0x564b53902370 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564b53822710 con 0x564b53907ed0 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564b5390afa0 con 0x564b53902d60 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77af80c740 10 client.0 _ll_register_callbacks cb 0x564b5381daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b5390c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b5390cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b5390c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40986/0 (socket says 172.21.15.46:40986) 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.444+0000 7f77a5e00640 1 -- 172.21.15.46:0/1315109489 learned_addr learned my addr 172.21.15.46:0/1315109489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a6800640 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53907ed0 0x564b53913d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a5400640 1 -- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53907ed0 msgr2=0x564b53913d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a5400640 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53907ed0 0x564b53913d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a5400640 1 -- 172.21.15.46:0/1315109489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 msgr2=0x564b5390c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a5400640 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b5390c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a5400640 1 -- 172.21.15.46:0/1315109489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5357b3f0 con 0x564b53902d60 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b5390c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f77a5400640 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b5390cd30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7798005f00 tx=0x7f7798033e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad87353559528127 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.445+0000 7f778b400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f778b400640 1 -- 172.21.15.46:0/1315109489 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779804c030 con 0x564b53902d60 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7798033c80 con 0x564b53902d60 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f778b400640 1 -- 172.21.15.46:0/1315109489 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f779802c900 con 0x564b53902d60 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f778b400640 1 -- 172.21.15.46:0/1315109489 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7798059050 con 0x564b53902d60 2024-04-17T07:59:23.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f77af80c740 10 client.25672 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564b5353b0f0 con 0x564b53902d60 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f778b400640 1 -- 172.21.15.46:0/1315109489 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798033c80 con 0x564b53902d60 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.446+0000 7f778b400640 1 -- 172.21.15.46:0/1315109489 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f7798004530 con 0x564b53902d60 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.447+0000 7f77af80c740 10 client.25672 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.447+0000 7f77af80c740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.447+0000 7f77af80c740 20 client.25672 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.447+0000 7f77af80c740 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53918260 0x564b5391a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.447+0000 7f77af80c740 4 client.25672 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.447+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b53876b40 con 0x564b53918260 2024-04-17T07:59:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.447+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53918260 0x564b5391a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.448+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53918260 0x564b5391a650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b892a7aa6639674 server_cookie=3b82468a5dc203e0 in_seq=0 out_seq=0 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.448+0000 7f778b400640 10 client.25672 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f778b400640 1 -- 172.21.15.46:0/1315109489 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b53876b40 con 0x564b53918260 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f778b400640 10 client.25672 handle_command_reply: tid=0 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 client.25672 shutdown 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 msgr2=0x564b5390cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b5390cd30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7798005f00 tx=0x7f7798033e30 comp rx=0 tx=0).stop 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 shutdown_connections 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53918260 0x564b5391a650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53907ed0 0x564b53913d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b5390cd30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 --2- 172.21.15.46:0/1315109489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53902370 0x564b5390c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f778b400640 0 client.25672 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 >> 172.21.15.46:0/1315109489 conn(0x564b538fd8e0 msgr2=0x564b538fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 shutdown_connections 2024-04-17T07:59:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.449+0000 7f77af80c740 1 -- 172.21.15.46:0/1315109489 wait complete. 2024-04-17T07:59:23.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.450+0000 7f77af80c740 20 client.25672 trim_cache size 0 max 16384 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 Processor -- start 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 -- start start 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b538fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53913d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b538fc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81af570 con 0x564b53907ed0 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81ace30 con 0x564b53902370 2024-04-17T07:59:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.469+0000 7f77af80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81aac80 con 0x564b53902d60 2024-04-17T07:59:23.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b538fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b538fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b538fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:33082/0 (socket says 172.21.15.46:33082) 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 -- 172.21.15.46:0/1291791494 learned_addr learned my addr 172.21.15.46:0/1291791494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b538fc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41002/0 (socket says 172.21.15.46:41002) 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5400640 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53913d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 -- 172.21.15.46:0/1291791494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 msgr2=0x564b53913d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53913d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 -- 172.21.15.46:0/1291791494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 msgr2=0x564b538fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b538fc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 -- 172.21.15.46:0/1291791494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b538fe650 con 0x564b53902370 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f77a5e00640 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b538fc250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77900993e0 tx=0x7f77900ce100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c16c1eba3dd760 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.470+0000 7f7796600640 1 -- 172.21.15.46:0/1291791494 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779009c7e0 con 0x564b53902370 2024-04-17T07:59:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f7796600640 1 -- 172.21.15.46:0/1291791494 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779009d110 con 0x564b53902370 2024-04-17T07:59:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f7796600640 1 -- 172.21.15.46:0/1291791494 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779009c370 con 0x564b53902370 2024-04-17T07:59:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 -- 172.21.15.46:0/1291791494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 msgr2=0x564b538fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b538fc250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77900993e0 tx=0x7f77900ce100 comp rx=0 tx=0).stop 2024-04-17T07:59:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 -- 172.21.15.46:0/1291791494 shutdown_connections 2024-04-17T07:59:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b538fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53913d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 --2- 172.21.15.46:0/1291791494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b538fc250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 -- 172.21.15.46:0/1291791494 >> 172.21.15.46:0/1291791494 conn(0x564b538fd8e0 msgr2=0x564b53904220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 -- 172.21.15.46:0/1291791494 shutdown_connections 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.471+0000 7f77af80c740 1 -- 172.21.15.46:0/1291791494 wait complete. 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 -- start start 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 Processor -- start 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b539115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53911b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b53912b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81ab2f0 con 0x564b53907ed0 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81ac760 con 0x564b53902370 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81ab9c0 con 0x564b53902d60 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.472+0000 7f77af80c740 10 client.0 _ll_register_callbacks cb 0x7f77a0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53911b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53911b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50878/0 (socket says 172.21.15.46:50878) 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 -- 172.21.15.46:0/975384547 learned_addr learned my addr 172.21.15.46:0/975384547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b53912b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5e00640 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b539115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 -- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 msgr2=0x564b539115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b539115d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 -- 172.21.15.46:0/975384547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 msgr2=0x564b53912b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b53912b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 -- 172.21.15.46:0/975384547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5390b0e0 con 0x564b53902d60 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f77a5400640 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53911b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7798000f80 tx=0x7f7798009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=499d830ee199e172 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.473+0000 7f778b400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.474+0000 7f778b400640 1 -- 172.21.15.46:0/975384547 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779804c070 con 0x564b53902d60 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.474+0000 7f778b400640 1 -- 172.21.15.46:0/975384547 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f779802ca90 con 0x564b53902d60 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.474+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779009d110 con 0x564b53902d60 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.474+0000 7f778b400640 1 -- 172.21.15.46:0/975384547 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7798043070 con 0x564b53902d60 2024-04-17T07:59:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.474+0000 7f77af80c740 10 client.25678 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.474+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564b53986f00 con 0x564b53902d60 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f778b400640 1 -- 172.21.15.46:0/975384547 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779009d110 con 0x564b53902d60 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f778b400640 1 -- 172.21.15.46:0/975384547 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f7798003d50 con 0x564b53902d60 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f77af80c740 10 client.25678 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f77af80c740 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f77af80c740 20 client.25678 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f77af80c740 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53987780 0x564b53989ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f77af80c740 4 client.25678 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b538709b0 con 0x564b53987780 2024-04-17T07:59:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.475+0000 7f77a5e00640 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53987780 0x564b53989ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77a5e00640 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53987780 0x564b53989ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817cdbdfd29d3d78 server_cookie=23831fc45d292261 in_seq=0 out_seq=0 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f778b400640 10 client.25678 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f778b400640 1 -- 172.21.15.46:0/975384547 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564b538709b0 con 0x564b53987780 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f778b400640 10 client.25678 handle_command_reply: tid=0 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 client.25678 shutdown 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 msgr2=0x564b53911b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53911b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7798000f80 tx=0x7f7798009930 comp rx=0 tx=0).stop 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 shutdown_connections 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x564b53987780 0x564b53989ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b53907ed0 0x564b53912b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564b53902d60 0x564b53911b10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 --2- 172.21.15.46:0/975384547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b53902370 0x564b539115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 >> 172.21.15.46:0/975384547 conn(0x564b538fd8e0 msgr2=0x564b53900b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f778b400640 0 client.25678 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 shutdown_connections 2024-04-17T07:59:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.476+0000 7f77af80c740 1 -- 172.21.15.46:0/975384547 wait complete. 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77af80c740 20 client.25678 trim_cache size 0 max 16384 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "ab9ab8b1-fb93-40da-a0ea-f70b0a52dc8c": { 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "ab9ab8b1-fb93-40da-a0ea-f70b0a52dc8c", 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:59:23.480 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:59:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f778c0a3bb0 msgr2=0x7f778c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f778c0a3bb0 0x7f778c0a6070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x564b539035c0 tx=0x7f77a0005f90 comp rx=0 tx=0).stop 2024-04-17T07:59:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 msgr2=0x7f77a819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 0x7f77a819a370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f779000e9f0 tx=0x7f779000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 shutdown_connections 2024-04-17T07:59:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f778c0a3bb0 0x7f778c0a6070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77a81039a0 0x7f77a81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77a8102fe0 0x7f77a819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 --2- 172.21.15.46:0/772708346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a8101df0 0x7f77a819a370 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 >> 172.21.15.46:0/772708346 conn(0x7f77a80fd360 msgr2=0x7f77a8074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 shutdown_connections 2024-04-17T07:59:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:23.477+0000 7f77a7200640 1 -- 172.21.15.46:0/772708346 wait complete. 2024-04-17T07:59:23.495 INFO:tasks.cephfs.filesystem:scrub status for tag:ab9ab8b1-fb93-40da-a0ea-f70b0a52dc8c - {'path': '/', 'tag': 'ab9ab8b1-fb93-40da-a0ea-f70b0a52dc8c', 'options': 'recursive,force'} 2024-04-17T07:59:25.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:24 smithi062 ceph-mon[36507]: scrub queued for path: / 2024-04-17T07:59:25.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:24 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:25.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:24 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T07:59:25.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:24 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:25.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:24 smithi062 ceph-mon[36507]: pgmap v444: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-17T07:59:25.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:24 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:59:25.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:24 smithi046 ceph-mon[32038]: scrub queued for path: / 2024-04-17T07:59:25.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:24 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:25.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:24 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T07:59:25.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:24 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:25.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:24 smithi046 ceph-mon[32038]: pgmap v444: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-17T07:59:25.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:24 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:59:25.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:24 smithi053 ceph-mon[36437]: scrub queued for path: / 2024-04-17T07:59:25.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:24 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:25.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:24 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T07:59:25.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:24 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:25.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:24 smithi053 ceph-mon[36437]: pgmap v444: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-17T07:59:25.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:24 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:59:26.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:25 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T07:59:26.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:59:26.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:59:26.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:25 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:59:26.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:25 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T07:59:26.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:59:26.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:59:26.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:25 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:59:26.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:25 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T07:59:26.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:59:26.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:59:26.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:25 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T07:59:27.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:27 smithi053 ceph-mon[36437]: pgmap v445: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-17T07:59:27.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:27 smithi062 ceph-mon[36507]: pgmap v445: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-17T07:59:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:27 smithi046 ceph-mon[32038]: pgmap v445: 129 pgs: 129 active+clean; 654 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-17T07:59:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:29 smithi053 ceph-mon[36437]: pgmap v446: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T07:59:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:29 smithi062 ceph-mon[36507]: pgmap v446: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T07:59:29.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:29 smithi046 ceph-mon[32038]: pgmap v446: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-17T07:59:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:31 smithi053 ceph-mon[36437]: pgmap v447: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-04-17T07:59:31.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:31 smithi062 ceph-mon[36507]: pgmap v447: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-04-17T07:59:31.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:31 smithi046 ceph-mon[32038]: pgmap v447: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-04-17T07:59:33.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:33 smithi053 ceph-mon[36437]: pgmap v448: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:33.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:33 smithi062 ceph-mon[36507]: pgmap v448: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:33.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:33 smithi046 ceph-mon[32038]: pgmap v448: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:35.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:35 smithi053 ceph-mon[36437]: pgmap v449: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:35.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:35 smithi062 ceph-mon[36507]: pgmap v449: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:35.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:35 smithi046 ceph-mon[32038]: pgmap v449: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T07:59:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:37 smithi053 ceph-mon[36437]: pgmap v450: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-17T07:59:37.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:37 smithi062 ceph-mon[36507]: pgmap v450: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-17T07:59:37.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:37 smithi046 ceph-mon[32038]: pgmap v450: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-17T07:59:39.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:39 smithi053 ceph-mon[36437]: pgmap v451: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-17T07:59:39.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:39 smithi062 ceph-mon[36507]: pgmap v451: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-17T07:59:39.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:39 smithi046 ceph-mon[32038]: pgmap v451: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-17T07:59:41.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:41 smithi053 ceph-mon[36437]: pgmap v452: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T07:59:41.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:41 smithi062 ceph-mon[36507]: pgmap v452: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T07:59:41.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:41 smithi046 ceph-mon[32038]: pgmap v452: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-17T07:59:43.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:43 smithi053 ceph-mon[36437]: pgmap v453: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-17T07:59:43.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:43 smithi062 ceph-mon[36507]: pgmap v453: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-17T07:59:43.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:43 smithi046 ceph-mon[32038]: pgmap v453: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-17T07:59:45.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:45 smithi062 ceph-mon[36507]: pgmap v454: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:45.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:45 smithi046 ceph-mon[32038]: pgmap v454: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:45.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:45 smithi053 ceph-mon[36437]: pgmap v454: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:47.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:47 smithi062 ceph-mon[36507]: pgmap v455: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:47.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:47 smithi046 ceph-mon[32038]: pgmap v455: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:47.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:47 smithi053 ceph-mon[36437]: pgmap v455: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:49.523 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:49 smithi053 ceph-mon[36437]: pgmap v456: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:49.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:49 smithi062 ceph-mon[36507]: pgmap v456: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:49.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:49 smithi046 ceph-mon[32038]: pgmap v456: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T07:59:51.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:51 smithi062 ceph-mon[36507]: pgmap v457: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:51.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:51 smithi046 ceph-mon[32038]: pgmap v457: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:51.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:51 smithi053 ceph-mon[36437]: pgmap v457: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:53.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:59:53.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:59:53.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:53 smithi062 ceph-mon[36507]: pgmap v458: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:59:53.586 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:53 smithi046 ceph-mon[32038]: pgmap v458: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:59:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 -- 172.21.15.46:0/1534742129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14074940 msgr2=0x7fda14074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 --2- 172.21.15.46:0/1534742129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14074940 0x7fda14074da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda00009900 tx=0x7fda0002bcc0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 -- 172.21.15.46:0/1534742129 shutdown_connections 2024-04-17T07:59:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 --2- 172.21.15.46:0/1534742129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda14109ca0 0x7fda1410c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 --2- 172.21.15.46:0/1534742129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14074940 0x7fda14074da0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 --2- 172.21.15.46:0/1534742129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda14075f20 0x7fda14074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 -- 172.21.15.46:0/1534742129 >> 172.21.15.46:0/1534742129 conn(0x7fda140fd360 msgr2=0x7fda140ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 -- 172.21.15.46:0/1534742129 shutdown_connections 2024-04-17T07:59:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.712+0000 7fda13200640 1 -- 172.21.15.46:0/1534742129 wait complete. 2024-04-17T07:59:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.713+0000 7fda13200640 1 Processor -- start 2024-04-17T07:59:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda13200640 1 -- start start 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda13200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda14074940 0x7fda1419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda13200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14075f20 0x7fda1419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda13200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda14109ca0 0x7fda141a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda13200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda1410fce0 con 0x7fda14075f20 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda13200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda1410fb60 con 0x7fda14074940 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda13200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda1410fe60 con 0x7fda14109ca0 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14075f20 0x7fda1419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda14074940 0x7fda1419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14075f20 0x7fda1419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55938/0 (socket says 172.21.15.46:55938) 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.714+0000 7fda11e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda14074940 0x7fda1419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53278/0 (socket says 172.21.15.46:53278) 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.715+0000 7fda11400640 1 -- 172.21.15.46:0/720919541 learned_addr learned my addr 172.21.15.46:0/720919541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.715+0000 7fda12800640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda14109ca0 0x7fda141a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.715+0000 7fda11400640 1 -- 172.21.15.46:0/720919541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda14109ca0 msgr2=0x7fda141a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.715+0000 7fda11400640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda14109ca0 0x7fda141a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.715+0000 7fda11400640 1 -- 172.21.15.46:0/720919541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda14074940 msgr2=0x7fda1419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.715+0000 7fda11400640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda14074940 0x7fda1419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.715+0000 7fda11400640 1 -- 172.21.15.46:0/720919541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda141a1ed0 con 0x7fda14075f20 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.716+0000 7fda11400640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14075f20 0x7fda1419a7f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fda00000c00 tx=0x7fda0002c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.716+0000 7fda0aa00640 1 -- 172.21.15.46:0/720919541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00043020 con 0x7fda14075f20 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.716+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda141a2160 con 0x7fda14075f20 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.716+0000 7fda0aa00640 1 -- 172.21.15.46:0/720919541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda000054e0 con 0x7fda14075f20 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.716+0000 7fda0aa00640 1 -- 172.21.15.46:0/720919541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00003c20 con 0x7fda14075f20 2024-04-17T07:59:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.716+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141a2700 con 0x7fda14075f20 2024-04-17T07:59:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.720+0000 7fda1be4d740 1 Processor -- start 2024-04-17T07:59:53.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.720+0000 7fda1be4d740 1 -- start start 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x564817895fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x56481789b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56481789b6e0 0x56481789db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda1be4d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5648177f4120 con 0x56481789b6e0 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda1be4d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56481779a340 con 0x564817895b80 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda1be4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5648177f00e0 con 0x564817896570 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda12800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x564817895fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda12800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x564817895fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53280/0 (socket says 172.21.15.46:53280) 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda12800640 1 -- 172.21.15.46:0/412398169 learned_addr learned my addr 172.21.15.46:0/412398169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.721+0000 7fda11e00640 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x56481789b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda0aa00640 1 -- 172.21.15.46:0/720919541 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fda00046020 con 0x7fda14075f20 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda11e00640 1 -- 172.21.15.46:0/412398169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 msgr2=0x564817895fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:53.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda11e00640 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x564817895fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda11e00640 1 -- 172.21.15.46:0/412398169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56481789b6e0 msgr2=0x56481789db70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:59:53.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:53 smithi053 ceph-mon[36437]: pgmap v458: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T07:59:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda11400640 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56481789b6e0 0x56481789db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda11e00640 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56481789b6e0 0x56481789db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda11e00640 1 -- 172.21.15.46:0/412398169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648175d07d0 con 0x564817896570 2024-04-17T07:59:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.722+0000 7fda11400640 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56481789b6e0 0x56481789db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:59:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.723+0000 7fda11e00640 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x56481789b070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda04009c20 tx=0x7fda04011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a870a8e6570a79d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.723+0000 7fd9ff400640 1 -- 172.21.15.46:0/412398169 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04009dd0 con 0x564817896570 2024-04-17T07:59:53.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.723+0000 7fd9ff400640 1 -- 172.21.15.46:0/412398169 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04005020 con 0x564817896570 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.723+0000 7fd9ff400640 1 -- 172.21.15.46:0/412398169 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04004b00 con 0x564817896570 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.724+0000 7fda0aa00640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd9f80a3c30 0x7fd9f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.724+0000 7fda1be4d740 1 -- 172.21.15.46:0/412398169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 msgr2=0x56481789b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.724+0000 7fda1be4d740 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x56481789b070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fda04009c20 tx=0x7fda04011810 comp rx=0 tx=0).stop 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.724+0000 7fda0aa00640 1 -- 172.21.15.46:0/720919541 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fda000f2760 con 0x7fda14075f20 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.724+0000 7fda1be4d740 1 -- 172.21.15.46:0/412398169 shutdown_connections 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.725+0000 7fda1be4d740 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56481789b6e0 0x56481789db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.725+0000 7fda1be4d740 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x56481789b070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.725+0000 7fda1be4d740 1 --2- 172.21.15.46:0/412398169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x564817895fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.725+0000 7fda1be4d740 1 -- 172.21.15.46:0/412398169 >> 172.21.15.46:0/412398169 conn(0x5648178888e0 msgr2=0x564817888ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.724+0000 7fda12800640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd9f80a3c30 0x7fd9f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.725+0000 7fda1be4d740 1 -- 172.21.15.46:0/412398169 shutdown_connections 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.725+0000 7fda1be4d740 1 -- 172.21.15.46:0/412398169 wait complete. 2024-04-17T07:59:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.725+0000 7fda1be4d740 1 -- start start 2024-04-17T07:59:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda12800640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd9f80a3c30 0x7fd9f80a60f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005fd0 tx=0x7fda0c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 1 Processor -- start 2024-04-17T07:59:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x56481788ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817896570 0x56481788f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817894470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56481788d470 con 0x564817896570 2024-04-17T07:59:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5648177ad710 con 0x564817895b80 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56481788d5f0 con 0x56481789b6e0 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817896570 0x56481788f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817896570 0x56481788f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55958/0 (socket says 172.21.15.46:55958) 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 -- 172.21.15.46:0/3397018825 learned_addr learned my addr 172.21.15.46:0/3397018825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11e00640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x56481788ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.726+0000 7fda1be4d740 10 client.0 _ll_register_callbacks cb 0x5648177a8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda12800640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817894470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 -- 172.21.15.46:0/3397018825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 msgr2=0x564817894470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817894470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 -- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 msgr2=0x56481788ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x56481788ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 -- 172.21.15.46:0/3397018825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648175063f0 con 0x564817896570 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda12800640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817894470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda04005020 con 0x564817896570 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.727+0000 7fda11400640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817896570 0x56481788f440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda000be470 tx=0x7fda000036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6002c7cb2de604 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.728+0000 7fd9f3e00640 10 client.15570 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.728+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3397018825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00004c80 con 0x564817896570 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.728+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3397018825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fda000f2450 con 0x564817896570 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.728+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3397018825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fda00005690 con 0x564817896570 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.728+0000 7fda1be4d740 10 client.15570 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.728+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=24}) -- 0x5648174c60f0 con 0x564817896570 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.728+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3397018825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fda000bade0 con 0x564817896570 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.729+0000 7fda1be4d740 10 client.15570 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.729+0000 7fda1be4d740 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.729+0000 7fda1be4d740 20 client.15570 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.729+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5648178b4470 0x5648178b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.729+0000 7fda1be4d740 4 client.15570 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:53.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.729+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564817801b40 con 0x5648178b4470 2024-04-17T07:59:53.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.729+0000 7fda11e00640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5648178b4470 0x5648178b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.730+0000 7fd9f3e00640 10 client.15570 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.730+0000 7fda11e00640 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5648178b4470 0x5648178b6900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32b446aee874ce7 server_cookie=2426973bb28bd214 in_seq=0 out_seq=0 2024-04-17T07:59:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3397018825 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564817801b40 con 0x5648178b4470 2024-04-17T07:59:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fd9f3e00640 10 client.15570 handle_command_reply: tid=0 2024-04-17T07:59:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 client.15570 shutdown 2024-04-17T07:59:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817896570 msgr2=0x56481788f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817896570 0x56481788f440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda000be470 tx=0x7fda000036f0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 shutdown_connections 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5648178b4470 0x5648178b6900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817894470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817896570 0x56481788f440 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3397018825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817895b80 0x56481788ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fd9f3e00640 0 client.15570 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 >> 172.21.15.46:0/3397018825 conn(0x5648178888e0 msgr2=0x564817887ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 shutdown_connections 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.731+0000 7fda1be4d740 1 -- 172.21.15.46:0/3397018825 wait complete. 2024-04-17T07:59:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.732+0000 7fda1be4d740 20 client.15570 trim_cache size 0 max 16384 2024-04-17T07:59:53.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.750+0000 7fda1be4d740 1 Processor -- start 2024-04-17T07:59:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.750+0000 7fda1be4d740 1 -- start start 2024-04-17T07:59:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x564817887250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817896570 0x564817887790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817887cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda1be4d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda141ab110 con 0x564817895b80 2024-04-17T07:59:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda1be4d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda141a89d0 con 0x564817896570 2024-04-17T07:59:53.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda1be4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda141a6820 con 0x56481789b6e0 2024-04-17T07:59:53.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x564817887250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x564817887250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55972/0 (socket says 172.21.15.46:55972) 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda11e00640 1 -- 172.21.15.46:0/929083811 learned_addr learned my addr 172.21.15.46:0/929083811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda11400640 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817896570 0x564817887790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.751+0000 7fda12800640 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817887cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda11e00640 1 -- 172.21.15.46:0/929083811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 msgr2=0x564817887cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda11e00640 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817887cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda11e00640 1 -- 172.21.15.46:0/929083811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817896570 msgr2=0x564817887790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda11e00640 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817896570 0x564817887790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda11e00640 1 -- 172.21.15.46:0/929083811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481788f790 con 0x564817895b80 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda11400640 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817896570 0x564817887790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda12800640 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817887cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fda11e00640 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x564817887250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda04009c20 tx=0x7fda040096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8542787e08e1a2a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.752+0000 7fd9fea00640 1 -- 172.21.15.46:0/929083811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040053a0 con 0x564817895b80 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.753+0000 7fd9fea00640 1 -- 172.21.15.46:0/929083811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04004520 con 0x564817895b80 2024-04-17T07:59:53.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.753+0000 7fd9fea00640 1 -- 172.21.15.46:0/929083811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04004cf0 con 0x564817895b80 2024-04-17T07:59:53.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.753+0000 7fda1be4d740 1 -- 172.21.15.46:0/929083811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 msgr2=0x564817887250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.753+0000 7fda1be4d740 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x564817887250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda04009c20 tx=0x7fda040096a0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 -- 172.21.15.46:0/929083811 shutdown_connections 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56481789b6e0 0x564817887cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564817896570 0x564817887790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 --2- 172.21.15.46:0/929083811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x564817887250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 -- 172.21.15.46:0/929083811 >> 172.21.15.46:0/929083811 conn(0x5648178888e0 msgr2=0x564817897ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 -- 172.21.15.46:0/929083811 shutdown_connections 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 -- 172.21.15.46:0/929083811 wait complete. 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.754+0000 7fda1be4d740 1 -- start start 2024-04-17T07:59:53.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 1 Processor -- start 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x5648178c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x5648178c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56481789b6e0 0x5648178c7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda141a6e90 con 0x564817895b80 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda141a8300 con 0x56481789b6e0 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fda141a7560 con 0x564817896570 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda1be4d740 10 client.0 _ll_register_callbacks cb 0x7fda000035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x5648178c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x5648178c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55974/0 (socket says 172.21.15.46:55974) 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.755+0000 7fda11e00640 1 -- 172.21.15.46:0/3390485296 learned_addr learned my addr 172.21.15.46:0/3390485296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:53.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.756+0000 7fda11400640 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x5648178c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.756+0000 7fda12800640 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56481789b6e0 0x5648178c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.756+0000 7fda11e00640 1 -- 172.21.15.46:0/3390485296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 msgr2=0x5648178c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.756+0000 7fda11e00640 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x5648178c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.756+0000 7fda11e00640 1 -- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56481789b6e0 msgr2=0x5648178c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.756+0000 7fda11e00640 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56481789b6e0 0x5648178c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.756+0000 7fda11e00640 1 -- 172.21.15.46:0/3390485296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648178c7750 con 0x564817895b80 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fda11e00640 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x5648178c5c10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fda04009460 tx=0x7fda04011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c319652e676cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fd9f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3390485296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0403a020 con 0x564817895b80 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3390485296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fda040053a0 con 0x564817895b80 2024-04-17T07:59:53.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda04004520 con 0x564817895b80 2024-04-17T07:59:53.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3390485296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fda04032040 con 0x564817895b80 2024-04-17T07:59:53.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fda1be4d740 10 client.15576 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:53.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.757+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56481791acf0 con 0x564817895b80 2024-04-17T07:59:53.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3390485296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04004520 con 0x564817895b80 2024-04-17T07:59:53.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3390485296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fda04035030 con 0x564817895b80 2024-04-17T07:59:53.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fda1be4d740 10 client.15576 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fda1be4d740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fda1be4d740 20 client.15576 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56481791b520 0x56481791d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fda1be4d740 4 client.15576 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.758+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648177fb9b0 con 0x56481791b520 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.759+0000 7fda11400640 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56481791b520 0x56481791d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.759+0000 7fda11400640 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56481791b520 0x56481791d940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb43bb144a56792 server_cookie=c98c17247259bd47 in_seq=0 out_seq=0 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.759+0000 7fd9f3e00640 10 client.15576 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.759+0000 7fd9f3e00640 1 -- 172.21.15.46:0/3390485296 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5648177fb9b0 con 0x56481791b520 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.759+0000 7fd9f3e00640 10 client.15576 handle_command_reply: tid=0 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.759+0000 7fda1be4d740 1 client.15576 shutdown 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 msgr2=0x5648178c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x5648178c5c10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fda04009460 tx=0x7fda04011e50 comp rx=0 tx=0).stop 2024-04-17T07:59:53.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 shutdown_connections 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56481791b520 0x56481791d940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56481789b6e0 0x5648178c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564817896570 0x5648178c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 --2- 172.21.15.46:0/3390485296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564817895b80 0x5648178c5c10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 >> 172.21.15.46:0/3390485296 conn(0x5648178888e0 msgr2=0x56481788bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fd9f3e00640 0 client.15576 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 shutdown_connections 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 1 -- 172.21.15.46:0/3390485296 wait complete. 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.760+0000 7fda1be4d740 20 client.15576 trim_cache size 0 max 16384 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:59:53.761 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-17T07:59:53.762 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-17T07:59:53.762 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:59:53.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd9f80a3c30 msgr2=0x7fd9f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd9f80a3c30 0x7fd9f80a60f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005fd0 tx=0x7fda0c009500 comp rx=0 tx=0).stop 2024-04-17T07:59:53.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14075f20 msgr2=0x7fda1419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14075f20 0x7fda1419a7f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fda00000c00 tx=0x7fda0002c6e0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 shutdown_connections 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd9f80a3c30 0x7fd9f80a60f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda14109ca0 0x7fda141a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda14075f20 0x7fda1419a7f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 --2- 172.21.15.46:0/720919541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda14074940 0x7fda1419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 >> 172.21.15.46:0/720919541 conn(0x7fda140fd360 msgr2=0x7fda14105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 shutdown_connections 2024-04-17T07:59:53.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.761+0000 7fda13200640 1 -- 172.21.15.46:0/720919541 wait complete. 2024-04-17T07:59:53.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:59:53.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:59:53.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 -- 172.21.15.46:0/1457457087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 msgr2=0x7fdd4010c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 --2- 172.21.15.46:0/1457457087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 0x7fdd4010c500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009ca0 tx=0x7fdd3802c100 comp rx=0 tx=0).stop 2024-04-17T07:59:53.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 -- 172.21.15.46:0/1457457087 shutdown_connections 2024-04-17T07:59:53.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 --2- 172.21.15.46:0/1457457087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 0x7fdd4010c500 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 --2- 172.21.15.46:0/1457457087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd401052b0 0x7fdd40105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 --2- 172.21.15.46:0/1457457087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd401040c0 0x7fdd401044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 -- 172.21.15.46:0/1457457087 >> 172.21.15.46:0/1457457087 conn(0x7fdd400ff630 msgr2=0x7fdd40101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 -- 172.21.15.46:0/1457457087 shutdown_connections 2024-04-17T07:59:53.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.972+0000 7fdd45200640 1 -- 172.21.15.46:0/1457457087 wait complete. 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.973+0000 7fdd45200640 1 Processor -- start 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.973+0000 7fdd45200640 1 -- start start 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd45200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd401040c0 0x7fdd4019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd45200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd401052b0 0x7fdd4019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd45200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 0x7fdd401a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd45200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4010fd10 con 0x7fdd401040c0 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd45200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4010fb90 con 0x7fdd401052b0 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd45200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4010fe90 con 0x7fdd40105c70 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd401040c0 0x7fdd4019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd401040c0 0x7fdd4019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55992/0 (socket says 172.21.15.46:55992) 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd3f400640 1 -- 172.21.15.46:0/3261667579 learned_addr learned my addr 172.21.15.46:0/3261667579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd3ea00640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd401052b0 0x7fdd4019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd3fe00640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 0x7fdd401a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.974+0000 7fdd3ea00640 1 -- 172.21.15.46:0/3261667579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 msgr2=0x7fdd401a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.975+0000 7fdd3ea00640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 0x7fdd401a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.975+0000 7fdd3ea00640 1 -- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd401040c0 msgr2=0x7fdd4019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.975+0000 7fdd3ea00640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd401040c0 0x7fdd4019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:53.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.975+0000 7fdd3ea00640 1 -- 172.21.15.46:0/3261667579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd401a1f40 con 0x7fdd401052b0 2024-04-17T07:59:53.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.975+0000 7fdd3ea00640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd401052b0 0x7fdd4019a860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00e9f0 tx=0x7fdd2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:53.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.975+0000 7fdd37e00640 1 -- 172.21.15.46:0/3261667579 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00ccc0 con 0x7fdd401052b0 2024-04-17T07:59:53.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.976+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd401a2230 con 0x7fdd401052b0 2024-04-17T07:59:53.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.976+0000 7fdd37e00640 1 -- 172.21.15.46:0/3261667579 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd2c010910 con 0x7fdd401052b0 2024-04-17T07:59:53.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.976+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd401a27d0 con 0x7fdd401052b0 2024-04-17T07:59:53.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.976+0000 7fdd37e00640 1 -- 172.21.15.46:0/3261667579 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c018960 con 0x7fdd401052b0 2024-04-17T07:59:53.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.977+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4010b380 con 0x7fdd401052b0 2024-04-17T07:59:53.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.981+0000 7fdd37e00640 1 -- 172.21.15.46:0/3261667579 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd2c010ac0 con 0x7fdd401052b0 2024-04-17T07:59:53.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.985+0000 7fdd37e00640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdd240a3ba0 0x7fdd240a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.986+0000 7fdd37e00640 1 -- 172.21.15.46:0/3261667579 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fdd2c0d10f0 con 0x7fdd401052b0 2024-04-17T07:59:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.986+0000 7fdd3f400640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdd240a3ba0 0x7fdd240a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.987+0000 7fdd37e00640 1 -- 172.21.15.46:0/3261667579 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdd2c0d1570 con 0x7fdd401052b0 2024-04-17T07:59:53.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:53.987+0000 7fdd3f400640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdd240a3ba0 0x7fdd240a6060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000bc80 tx=0x7fdd3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.422+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd4002d810 con 0x7fdd401052b0 2024-04-17T07:59:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.423+0000 7fdd37e00640 1 -- 172.21.15.46:0/3261667579 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7fdd2c09d6e0 con 0x7fdd401052b0 2024-04-17T07:59:54.422 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T07:59:54.423 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:58:39.019766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T07:59:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-04-17T07:59:54.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.426+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdd240a3ba0 msgr2=0x7fdd240a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.426+0000 7fdd45200640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdd240a3ba0 0x7fdd240a6060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000bc80 tx=0x7fdd3000b410 comp rx=0 tx=0).stop 2024-04-17T07:59:54.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd401052b0 msgr2=0x7fdd4019a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd401052b0 0x7fdd4019a860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00e9f0 tx=0x7fdd2c00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 shutdown_connections 2024-04-17T07:59:54.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fdd240a3ba0 0x7fdd240a6060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd40105c70 0x7fdd401a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd401052b0 0x7fdd4019a860 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 --2- 172.21.15.46:0/3261667579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd401040c0 0x7fdd4019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.427+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 >> 172.21.15.46:0/3261667579 conn(0x7fdd400ff630 msgr2=0x7fdd40108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.428+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 shutdown_connections 2024-04-17T07:59:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.428+0000 7fdd45200640 1 -- 172.21.15.46:0/3261667579 wait complete. 2024-04-17T07:59:54.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 -- 172.21.15.46:0/3184205738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 msgr2=0x7f76d4107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 --2- 172.21.15.46:0/3184205738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 0x7f76d4107740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c002bc40 comp rx=0 tx=0).stop 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 -- 172.21.15.46:0/3184205738 shutdown_connections 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 --2- 172.21.15.46:0/3184205738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4107c80 0x7f76d410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 --2- 172.21.15.46:0/3184205738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 0x7f76d4107740 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 --2- 172.21.15.46:0/3184205738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 0x7f76d4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 -- 172.21.15.46:0/3184205738 >> 172.21.15.46:0/3184205738 conn(0x7f76d40fc430 msgr2=0x7f76d40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 -- 172.21.15.46:0/3184205738 shutdown_connections 2024-04-17T07:59:54.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.685+0000 7f76d9e00640 1 -- 172.21.15.46:0/3184205738 wait complete. 2024-04-17T07:59:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.686+0000 7f76d9e00640 1 Processor -- start 2024-04-17T07:59:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.686+0000 7f76d9e00640 1 -- start start 2024-04-17T07:59:54.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.686+0000 7f76d9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 0x7f76d419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.686+0000 7f76d9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 0x7f76d419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.686+0000 7f76d9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4107c80 0x7f76d41a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d9e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410dc80 con 0x7f76d4105350 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d9e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410db00 con 0x7f76d4107c80 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d9e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410de00 con 0x7f76d4102a20 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 0x7f76d419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 0x7f76d419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4107c80 0x7f76d41a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 0x7f76d419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44724/0 (socket says 172.21.15.46:44724) 2024-04-17T07:59:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d2a00640 1 -- 172.21.15.46:0/2759742006 learned_addr learned my addr 172.21.15.46:0/2759742006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d2a00640 1 -- 172.21.15.46:0/2759742006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4107c80 msgr2=0x7f76d41a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.687+0000 7f76d2a00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4107c80 0x7f76d41a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d2a00640 1 -- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 msgr2=0x7f76d419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d2a00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 0x7f76d419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d2a00640 1 -- 172.21.15.46:0/2759742006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d41a2170 con 0x7f76d4102a20 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d2a00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 0x7f76d419a550 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76c400e9f0 tx=0x7f76c400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d8a00640 1 -- 172.21.15.46:0/2759742006 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c400cd80 con 0x7f76d4102a20 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d8a00640 1 -- 172.21.15.46:0/2759742006 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c4005500 con 0x7f76d4102a20 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d8a00640 1 -- 172.21.15.46:0/2759742006 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4010690 con 0x7f76d4102a20 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41a2460 con 0x7f76d4102a20 2024-04-17T07:59:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.688+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a2930 con 0x7f76d4102a20 2024-04-17T07:59:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.692+0000 7f76dca28740 1 Processor -- start 2024-04-17T07:59:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.692+0000 7f76dca28740 1 -- start start 2024-04-17T07:59:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76dca28740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a873c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76dca28740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58a8c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76dca28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a8ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76dca28740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555a589ed020 con 0x555a58a86fa0 2024-04-17T07:59:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76dca28740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a58993460 con 0x555a58a87990 2024-04-17T07:59:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76dca28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a589e8fe0 con 0x555a58a8cb00 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a873c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a873c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56026/0 (socket says 172.21.15.46:56026) 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.693+0000 7f76d3400640 1 -- 172.21.15.46:0/1501463091 learned_addr learned my addr 172.21.15.46:0/1501463091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d2000640 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a8ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d8a00640 1 -- 172.21.15.46:0/2759742006 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f76c40046c0 con 0x7f76d4102a20 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d3400640 1 -- 172.21.15.46:0/1501463091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 msgr2=0x555a58a8ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d3400640 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a8ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d3400640 1 -- 172.21.15.46:0/1501463091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 msgr2=0x555a58a8c490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:59:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d3400640 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58a8c490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d3400640 1 -- 172.21.15.46:0/1501463091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a5899b040 con 0x555a58a86fa0 2024-04-17T07:59:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.694+0000 7f76d3400640 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a873c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009fd0 tx=0x7f76cc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e324cdf183e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.695+0000 7f76bfe00640 1 -- 172.21.15.46:0/1501463091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc004d20 con 0x555a58a86fa0 2024-04-17T07:59:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.695+0000 7f76bfe00640 1 -- 172.21.15.46:0/1501463091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc004050 con 0x555a58a86fa0 2024-04-17T07:59:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.695+0000 7f76bfe00640 1 -- 172.21.15.46:0/1501463091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc005280 con 0x555a58a86fa0 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 -- 172.21.15.46:0/1501463091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 msgr2=0x555a58a873c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a873c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009fd0 tx=0x7f76cc0118d0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 -- 172.21.15.46:0/1501463091 shutdown_connections 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a8ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58a8c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 --2- 172.21.15.46:0/1501463091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a873c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 -- 172.21.15.46:0/1501463091 >> 172.21.15.46:0/1501463091 conn(0x555a58a82510 msgr2=0x555a58a82910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76d8a00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b80a3cb0 0x7f76b80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76d8a00640 1 -- 172.21.15.46:0/2759742006 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f76c40d0af0 con 0x7f76d4102a20 2024-04-17T07:59:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76dca28740 1 -- 172.21.15.46:0/1501463091 shutdown_connections 2024-04-17T07:59:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.696+0000 7f76d3400640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b80a3cb0 0x7f76b80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.697+0000 7f76dca28740 1 -- 172.21.15.46:0/1501463091 wait complete. 2024-04-17T07:59:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.697+0000 7f76dca28740 1 -- start start 2024-04-17T07:59:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.697+0000 7f76dca28740 1 Processor -- start 2024-04-17T07:59:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.697+0000 7f76d3400640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b80a3cb0 0x7f76b80a6170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76cc005fd0 tx=0x7f76cc000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76dca28740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76dca28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a87990 0x555a58a9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76dca28740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a8cb00 0x555a58aa1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76dca28740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555a58a8fab0 con 0x555a58a8cb00 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76dca28740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a589a6830 con 0x555a58a86fa0 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76dca28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555a58a8fc30 con 0x555a58a87990 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76dca28740 10 client.0 _ll_register_callbacks cb 0x555a58974fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76d2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76d2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a99d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53352/0 (socket says 172.21.15.46:53352) 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.698+0000 7f76d2a00640 1 -- 172.21.15.46:0/274437355 learned_addr learned my addr 172.21.15.46:0/274437355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2000640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a87990 0x555a58a9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d3400640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a8cb00 0x555a58aa1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2a00640 1 -- 172.21.15.46:0/274437355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a87990 msgr2=0x555a58a9a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2a00640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a87990 0x555a58a9a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2a00640 1 -- 172.21.15.46:0/274437355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a8cb00 msgr2=0x555a58aa1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2a00640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a8cb00 0x555a58aa1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2a00640 1 -- 172.21.15.46:0/274437355 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a589aa920 con 0x555a58a86fa0 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2000640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a87990 0x555a58a9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d3400640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a8cb00 0x555a58aa1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76d2a00640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a99d00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76c409c6e0 tx=0x7f76c4098db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7181982c20563d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.699+0000 7f76bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.700+0000 7f76bcc00640 1 -- 172.21.15.46:0/274437355 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40a53c0 con 0x555a58a86fa0 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.700+0000 7f76bcc00640 1 -- 172.21.15.46:0/274437355 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f76c40bbba0 con 0x555a58a86fa0 2024-04-17T07:59:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.700+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76cc004050 con 0x555a58a86fa0 2024-04-17T07:59:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.700+0000 7f76bcc00640 1 -- 172.21.15.46:0/274437355 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f76c409d0e0 con 0x555a58a86fa0 2024-04-17T07:59:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.700+0000 7f76dca28740 10 client.25727 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.700+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555a589787f0 con 0x555a58a86fa0 2024-04-17T07:59:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.700+0000 7f76bcc00640 1 -- 172.21.15.46:0/274437355 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc004050 con 0x555a58a86fa0 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76bcc00640 1 -- 172.21.15.46:0/274437355 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f76c409be30 con 0x555a58a86fa0 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76dca28740 10 client.25727 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76dca28740 10 client.25727 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76dca28740 20 client.25727 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76dca28740 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58ab67e0 0x555a58ab8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76dca28740 4 client.25727 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a589faa40 con 0x555a58ab67e0 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.701+0000 7f76d2000640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58ab67e0 0x555a58ab8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.702+0000 7f76bcc00640 10 client.25727 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.702+0000 7f76d2000640 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58ab67e0 0x555a58ab8c30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a796c26c971471f5 server_cookie=eb0d78105658e983 in_seq=0 out_seq=0 2024-04-17T07:59:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76bcc00640 1 -- 172.21.15.46:0/274437355 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a589faa40 con 0x555a58ab67e0 2024-04-17T07:59:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76bcc00640 10 client.25727 handle_command_reply: tid=0 2024-04-17T07:59:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 client.25727 shutdown 2024-04-17T07:59:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 msgr2=0x555a58a99d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a99d00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76c409c6e0 tx=0x7f76c4098db0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 shutdown_connections 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58ab67e0 0x555a58ab8c30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a8cb00 0x555a58aa1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a87990 0x555a58a9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 --2- 172.21.15.46:0/274437355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a99d00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 >> 172.21.15.46:0/274437355 conn(0x555a58a82510 msgr2=0x555a58a81710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.703+0000 7f76bcc00640 0 client.25727 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.704+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 shutdown_connections 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.704+0000 7f76dca28740 1 -- 172.21.15.46:0/274437355 wait complete. 2024-04-17T07:59:54.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.704+0000 7f76dca28740 20 client.25727 trim_cache size 0 max 16384 2024-04-17T07:59:54.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.718+0000 7f76dca28740 1 Processor -- start 2024-04-17T07:59:54.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.718+0000 7f76dca28740 1 -- start start 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76dca28740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a9a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76dca28740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58aa1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76dca28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a85e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76dca28740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41a2bf0 con 0x555a58a86fa0 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76dca28740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4071920 con 0x555a58a87990 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76dca28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76d406f770 con 0x555a58a8cb00 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a9a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a9a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56044/0 (socket says 172.21.15.46:56044) 2024-04-17T07:59:54.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58aa1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d2a00640 1 -- 172.21.15.46:0/577890222 learned_addr learned my addr 172.21.15.46:0/577890222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d3400640 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a85e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d2000640 1 -- 172.21.15.46:0/577890222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 msgr2=0x555a58a85e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d2000640 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a85e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.719+0000 7f76d2000640 1 -- 172.21.15.46:0/577890222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 msgr2=0x555a58a9a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.720+0000 7f76d2000640 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a9a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.720+0000 7f76d2000640 1 -- 172.21.15.46:0/577890222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a586ff3f0 con 0x555a58a87990 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.720+0000 7f76d2a00640 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a9a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.720+0000 7f76d2000640 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58aa1240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009230 tx=0x7f76c002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b0a8ee3983f829 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.720+0000 7f76bf400640 1 -- 172.21.15.46:0/577890222 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c004c070 con 0x555a58a87990 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.720+0000 7f76bf400640 1 -- 172.21.15.46:0/577890222 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c0048bc0 con 0x555a58a87990 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.720+0000 7f76bf400640 1 -- 172.21.15.46:0/577890222 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c00488d0 con 0x555a58a87990 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 -- 172.21.15.46:0/577890222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 msgr2=0x555a58aa1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58aa1240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009230 tx=0x7f76c002e040 comp rx=0 tx=0).stop 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 -- 172.21.15.46:0/577890222 shutdown_connections 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a85e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a87990 0x555a58aa1240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 --2- 172.21.15.46:0/577890222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a86fa0 0x555a58a9a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 -- 172.21.15.46:0/577890222 >> 172.21.15.46:0/577890222 conn(0x555a58a82510 msgr2=0x555a58a85770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 -- 172.21.15.46:0/577890222 shutdown_connections 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 -- 172.21.15.46:0/577890222 wait complete. 2024-04-17T07:59:54.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.721+0000 7f76dca28740 1 -- start start 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 1 Processor -- start 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a9e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a87990 0x555a58a9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a9faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76d406fde0 con 0x555a58a87990 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4071250 con 0x555a58a86fa0 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40704b0 con 0x555a58a8cb00 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76dca28740 10 client.0 _ll_register_callbacks cb 0x7f76c4004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.722+0000 7f76d2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a87990 0x555a58a9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a87990 0x555a58a9eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56048/0 (socket says 172.21.15.46:56048) 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 -- 172.21.15.46:0/3981514885 learned_addr learned my addr 172.21.15.46:0/3981514885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2a00640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a9e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d3400640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a9faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 -- 172.21.15.46:0/3981514885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 msgr2=0x555a58a9faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a9faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 -- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 msgr2=0x555a58a9e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a9e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 -- 172.21.15.46:0/3981514885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a587c97d0 con 0x555a58a87990 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2a00640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a9e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76d2000640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a87990 0x555a58a9eb80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f76c0048410 tx=0x7f76c0049ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9069bff35bcd7690 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.723+0000 7f76bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.724+0000 7f76bcc00640 1 -- 172.21.15.46:0/3981514885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0057070 con 0x555a58a87990 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.724+0000 7f76bcc00640 1 -- 172.21.15.46:0/3981514885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f76c0048810 con 0x555a58a87990 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.724+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c0048bc0 con 0x555a58a87990 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.724+0000 7f76bcc00640 1 -- 172.21.15.46:0/3981514885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f76c0063050 con 0x555a58a87990 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.724+0000 7f76dca28740 10 client.15597 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.724+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555a58aa1210 con 0x555a58a87990 2024-04-17T07:59:54.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.724+0000 7f76bcc00640 1 -- 172.21.15.46:0/3981514885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c0048bc0 con 0x555a58a87990 2024-04-17T07:59:54.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76bcc00640 1 -- 172.21.15.46:0/3981514885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f76c00046c0 con 0x555a58a87990 2024-04-17T07:59:54.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76dca28740 10 client.15597 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:54.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76dca28740 10 client.15597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T07:59:54.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76dca28740 20 client.15597 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:54.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76dca28740 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58b1d010 0x555a58b1f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76dca28740 4 client.15597 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:59:54.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a589f48b0 con 0x555a58b1d010 2024-04-17T07:59:54.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.725+0000 7f76d2a00640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58b1d010 0x555a58b1f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.726+0000 7f76d2a00640 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58b1d010 0x555a58b1f400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afbd6d4ebbb385a5 server_cookie=f1f291bd10b8742 in_seq=0 out_seq=0 2024-04-17T07:59:54.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.726+0000 7f76bcc00640 10 client.15597 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:54.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.726+0000 7f76bcc00640 1 -- 172.21.15.46:0/3981514885 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a589f48b0 con 0x555a58b1d010 2024-04-17T07:59:54.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.726+0000 7f76bcc00640 10 client.15597 handle_command_reply: tid=0 2024-04-17T07:59:54.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.726+0000 7f76dca28740 1 client.15597 shutdown 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a87990 msgr2=0x555a58a9eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a87990 0x555a58a9eb80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f76c0048410 tx=0x7f76c0049ee0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 shutdown_connections 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x555a58b1d010 0x555a58b1f400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555a58a8cb00 0x555a58a9faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a58a87990 0x555a58a9eb80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 --2- 172.21.15.46:0/3981514885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a58a86fa0 0x555a58a9e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76bcc00640 0 client.15597 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 >> 172.21.15.46:0/3981514885 conn(0x555a58a82510 msgr2=0x555a58a9a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 shutdown_connections 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 1 -- 172.21.15.46:0/3981514885 wait complete. 2024-04-17T07:59:54.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.727+0000 7f76dca28740 20 client.15597 trim_cache size 0 max 16384 2024-04-17T07:59:54.729 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T07:59:54.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b80a3cb0 msgr2=0x7f76b80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b80a3cb0 0x7f76b80a6170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76cc005fd0 tx=0x7f76cc000f40 comp rx=0 tx=0).stop 2024-04-17T07:59:54.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 msgr2=0x7f76d419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 0x7f76d419a550 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76c400e9f0 tx=0x7f76c400edf0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 shutdown_connections 2024-04-17T07:59:54.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f76b80a3cb0 0x7f76b80a6170 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4107c80 0x7f76d41a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d4105350 0x7f76d419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 --2- 172.21.15.46:0/2759742006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76d4102a20 0x7f76d419a550 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.728+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 >> 172.21.15.46:0/2759742006 conn(0x7f76d40fc430 msgr2=0x7f76d41021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.729+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 shutdown_connections 2024-04-17T07:59:54.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.729+0000 7f76d9e00640 1 -- 172.21.15.46:0/2759742006 wait complete. 2024-04-17T07:59:54.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:59:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.934+0000 7fd140e00640 1 -- 172.21.15.46:0/4193339722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c075f40 msgr2=0x7fd13c074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.934+0000 7fd140e00640 1 --2- 172.21.15.46:0/4193339722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c075f40 0x7fd13c074390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009860 tx=0x7fd12c02bc10 comp rx=0 tx=0).stop 2024-04-17T07:59:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.935+0000 7fd140e00640 1 -- 172.21.15.46:0/4193339722 shutdown_connections 2024-04-17T07:59:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.935+0000 7fd140e00640 1 --2- 172.21.15.46:0/4193339722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c109c80 0x7fd13c10c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.935+0000 7fd140e00640 1 --2- 172.21.15.46:0/4193339722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd13c074960 0x7fd13c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.935+0000 7fd140e00640 1 --2- 172.21.15.46:0/4193339722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c075f40 0x7fd13c074390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.935+0000 7fd140e00640 1 -- 172.21.15.46:0/4193339722 >> 172.21.15.46:0/4193339722 conn(0x7fd13c0fd360 msgr2=0x7fd13c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.935+0000 7fd140e00640 1 -- 172.21.15.46:0/4193339722 shutdown_connections 2024-04-17T07:59:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.935+0000 7fd140e00640 1 -- 172.21.15.46:0/4193339722 wait complete. 2024-04-17T07:59:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.936+0000 7fd140e00640 1 Processor -- start 2024-04-17T07:59:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.936+0000 7fd140e00640 1 -- start start 2024-04-17T07:59:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.936+0000 7fd140e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c074960 0x7fd13c105170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.936+0000 7fd140e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c075f40 0x7fd13c1056b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd140e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd13c109c80 0x7fd13c101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd140e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c10fc80 con 0x7fd13c109c80 2024-04-17T07:59:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd140e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c10fb00 con 0x7fd13c074960 2024-04-17T07:59:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd140e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c10fe00 con 0x7fd13c075f40 2024-04-17T07:59:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c074960 0x7fd13c105170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c074960 0x7fd13c105170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53386/0 (socket says 172.21.15.46:53386) 2024-04-17T07:59:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 -- 172.21.15.46:0/962556066 learned_addr learned my addr 172.21.15.46:0/962556066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13aa00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c075f40 0x7fd13c1056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13be00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd13c109c80 0x7fd13c101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 -- 172.21.15.46:0/962556066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c075f40 msgr2=0x7fd13c1056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c075f40 0x7fd13c1056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 -- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd13c109c80 msgr2=0x7fd13c101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd13c109c80 0x7fd13c101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13b400640 1 -- 172.21.15.46:0/962556066 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c1025a0 con 0x7fd13c074960 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.937+0000 7fd13be00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd13c109c80 0x7fd13c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.938+0000 7fd13b400640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c074960 0x7fd13c105170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd12c034a80 tx=0x7fd12c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.938+0000 7fd133e00640 1 -- 172.21.15.46:0/962556066 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c041030 con 0x7fd13c074960 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.938+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c1a6720 con 0x7fd13c074960 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.938+0000 7fd133e00640 1 -- 172.21.15.46:0/962556066 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12c02ca90 con 0x7fd13c074960 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.938+0000 7fd133e00640 1 -- 172.21.15.46:0/962556066 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c039040 con 0x7fd13c074960 2024-04-17T07:59:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.938+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c1a6ad0 con 0x7fd13c074960 2024-04-17T07:59:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.941+0000 7fd14502b740 1 Processor -- start 2024-04-17T07:59:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.942+0000 7fd14502b740 1 -- start start 2024-04-17T07:59:54.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.942+0000 7fd14502b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd967580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.942+0000 7fd14502b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967b50 0x55e4bd96c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.942+0000 7fd14502b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd96f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.942+0000 7fd14502b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4bd8cd020 con 0x55e4bd967b50 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.942+0000 7fd14502b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e4bd873460 con 0x55e4bd96ccc0 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.942+0000 7fd14502b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e4bd8c8fe0 con 0x55e4bd967160 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd96f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd967580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd96f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53390/0 (socket says 172.21.15.46:53390) 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd967580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44774/0 (socket says 172.21.15.46:44774) 2024-04-17T07:59:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 -- 172.21.15.46:0/621991723 learned_addr learned my addr 172.21.15.46:0/621991723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 -- 172.21.15.46:0/621991723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 msgr2=0x55e4bd967580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 --2- 172.21.15.46:0/621991723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd967580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 -- 172.21.15.46:0/621991723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967b50 msgr2=0x55e4bd96c650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:59:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 --2- 172.21.15.46:0/621991723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967b50 0x55e4bd96c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 -- 172.21.15.46:0/621991723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4bd87b040 con 0x55e4bd96ccc0 2024-04-17T07:59:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.943+0000 7fd13aa00640 1 --2- 172.21.15.46:0/621991723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd96f150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b740 tx=0x7fd1280045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9bef8be6a99918 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.944+0000 7fd130c00640 1 -- 172.21.15.46:0/621991723 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128003bd0 con 0x55e4bd96ccc0 2024-04-17T07:59:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.944+0000 7fd130c00640 1 -- 172.21.15.46:0/621991723 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd128015070 con 0x55e4bd96ccc0 2024-04-17T07:59:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.944+0000 7fd130c00640 1 -- 172.21.15.46:0/621991723 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1280106a0 con 0x55e4bd96ccc0 2024-04-17T07:59:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.944+0000 7fd14502b740 1 -- 172.21.15.46:0/621991723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 msgr2=0x55e4bd96f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.944+0000 7fd14502b740 1 --2- 172.21.15.46:0/621991723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd96f150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd12800b740 tx=0x7fd1280045f0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.945+0000 7fd14502b740 1 -- 172.21.15.46:0/621991723 shutdown_connections 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.945+0000 7fd14502b740 1 --2- 172.21.15.46:0/621991723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd96f150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.945+0000 7fd14502b740 1 --2- 172.21.15.46:0/621991723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967b50 0x55e4bd96c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.945+0000 7fd14502b740 1 --2- 172.21.15.46:0/621991723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd967580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.945+0000 7fd14502b740 1 -- 172.21.15.46:0/621991723 >> 172.21.15.46:0/621991723 conn(0x55e4bd9626d0 msgr2=0x55e4bd962ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.945+0000 7fd133e00640 1 -- 172.21.15.46:0/962556066 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd12c044070 con 0x7fd13c074960 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.946+0000 7fd14502b740 1 -- 172.21.15.46:0/621991723 shutdown_connections 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.946+0000 7fd14502b740 1 -- 172.21.15.46:0/621991723 wait complete. 2024-04-17T07:59:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.946+0000 7fd14502b740 1 -- start start 2024-04-17T07:59:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 1 Processor -- start 2024-04-17T07:59:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd133e00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1200a3b40 0x7fd1200a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd971800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd971d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd976d70 0x55e4bd979160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e4bd96fc70 con 0x55e4bd967160 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e4bd886830 con 0x55e4bd96ccc0 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e4bd96fdf0 con 0x55e4bd976d70 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd133e00640 1 -- 172.21.15.46:0/962556066 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd12c0f17b0 con 0x7fd13c074960 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.947+0000 7fd14502b740 10 client.0 _ll_register_callbacks cb 0x55e4bd854fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.948+0000 7fd13be00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1200a3b40 0x7fd1200a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.948+0000 7fd13be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd976d70 0x55e4bd979160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.948+0000 7fd13be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd976d70 0x55e4bd979160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44790/0 (socket says 172.21.15.46:44790) 2024-04-17T07:59:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.948+0000 7fd13be00640 1 -- 172.21.15.46:0/1559989957 learned_addr learned my addr 172.21.15.46:0/1559989957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.948+0000 7fd13b400640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd971800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.948+0000 7fd13aa00640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd971d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13be00640 1 -- 172.21.15.46:0/1559989957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 msgr2=0x55e4bd971d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13be00640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd971d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13be00640 1 -- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 msgr2=0x55e4bd971800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13be00640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd971800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13be00640 1 -- 172.21.15.46:0/1559989957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bd88a920 con 0x55e4bd976d70 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13aa00640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd971d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13b400640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd971800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.949+0000 7fd13be00640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd976d70 0x55e4bd979160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd134010d10 tx=0x7fd13400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bec3b6014ad00b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.950+0000 7fd125600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.950+0000 7fd13be00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1200a3b40 0x7fd1200a6000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd134009fd0 tx=0x7fd134013000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.950+0000 7fd125600640 1 -- 172.21.15.46:0/1559989957 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134024070 con 0x55e4bd976d70 2024-04-17T07:59:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.950+0000 7fd125600640 1 -- 172.21.15.46:0/1559989957 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd13401ea80 con 0x55e4bd976d70 2024-04-17T07:59:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.950+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd128015070 con 0x55e4bd976d70 2024-04-17T07:59:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.951+0000 7fd125600640 1 -- 172.21.15.46:0/1559989957 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd13402e050 con 0x55e4bd976d70 2024-04-17T07:59:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.951+0000 7fd14502b740 10 client.25732 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.951+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4bd8587f0 con 0x55e4bd976d70 2024-04-17T07:59:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.951+0000 7fd125600640 1 -- 172.21.15.46:0/1559989957 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd128015070 con 0x55e4bd976d70 2024-04-17T07:59:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.952+0000 7fd125600640 1 -- 172.21.15.46:0/1559989957 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fd134029440 con 0x55e4bd976d70 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.952+0000 7fd14502b740 10 client.25732 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.952+0000 7fd14502b740 10 client.25732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.952+0000 7fd14502b740 20 client.25732 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.952+0000 7fd14502b740 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd97d7a0 0x55e4bd97fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.953+0000 7fd14502b740 4 client.25732 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.953+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4bd8daa40 con 0x55e4bd97d7a0 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.953+0000 7fd13b400640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd97d7a0 0x55e4bd97fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.953+0000 7fd13b400640 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd97d7a0 0x55e4bd97fc00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=856f667193a205b9 server_cookie=3f4028b31d606291 in_seq=0 out_seq=0 2024-04-17T07:59:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.953+0000 7fd125600640 10 client.25732 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.954+0000 7fd125600640 1 -- 172.21.15.46:0/1559989957 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4bd8daa40 con 0x55e4bd97d7a0 2024-04-17T07:59:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.954+0000 7fd125600640 10 client.25732 handle_command_reply: tid=0 2024-04-17T07:59:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 client.25732 shutdown 2024-04-17T07:59:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd976d70 msgr2=0x55e4bd979160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd976d70 0x55e4bd979160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd134010d10 tx=0x7fd13400e720 comp rx=0 tx=0).stop 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 shutdown_connections 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd976d70 0x55e4bd979160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd971d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd97d7a0 0x55e4bd97fc00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 --2- 172.21.15.46:0/1559989957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd971800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd125600640 0 client.25732 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 >> 172.21.15.46:0/1559989957 conn(0x55e4bd9626d0 msgr2=0x55e4bd961930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 shutdown_connections 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.955+0000 7fd14502b740 1 -- 172.21.15.46:0/1559989957 wait complete. 2024-04-17T07:59:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.956+0000 7fd14502b740 20 client.25732 trim_cache size 0 max 16384 2024-04-17T07:59:54.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.984+0000 7fd14502b740 1 Processor -- start 2024-04-17T07:59:54.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.984+0000 7fd14502b740 1 -- start start 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd14502b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd9610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd14502b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd9615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd14502b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd976d70 0x55e4bd961b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd14502b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1b3900 con 0x55e4bd976d70 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd14502b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1b11c0 con 0x55e4bd96ccc0 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd14502b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1af010 con 0x55e4bd967160 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd13b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd9610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd13b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd9610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44796/0 (socket says 172.21.15.46:44796) 2024-04-17T07:59:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd13b400640 1 -- 172.21.15.46:0/1837482722 learned_addr learned my addr 172.21.15.46:0/1837482722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd13aa00640 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd9615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.985+0000 7fd13be00640 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd976d70 0x55e4bd961b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13b400640 1 -- 172.21.15.46:0/1837482722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 msgr2=0x55e4bd9615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13b400640 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd9615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13b400640 1 -- 172.21.15.46:0/1837482722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd976d70 msgr2=0x55e4bd961b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13b400640 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd976d70 0x55e4bd961b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13b400640 1 -- 172.21.15.46:0/1837482722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4bd5df3f0 con 0x55e4bd967160 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13be00640 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd976d70 0x55e4bd961b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13aa00640 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd9615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd13b400640 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd9610a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0bd400 tx=0x7fd12c02bc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac3d311799d1f528 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd127e00640 1 -- 172.21.15.46:0/1837482722 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0bd700 con 0x55e4bd967160 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd127e00640 1 -- 172.21.15.46:0/1837482722 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12c0c1ab0 con 0x55e4bd967160 2024-04-17T07:59:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.986+0000 7fd127e00640 1 -- 172.21.15.46:0/1837482722 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0c5ee0 con 0x55e4bd967160 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.987+0000 7fd14502b740 1 -- 172.21.15.46:0/1837482722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 msgr2=0x55e4bd9610a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.987+0000 7fd14502b740 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd9610a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0bd400 tx=0x7fd12c02bc10 comp rx=0 tx=0).stop 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 -- 172.21.15.46:0/1837482722 shutdown_connections 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd976d70 0x55e4bd961b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd96ccc0 0x55e4bd9615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 --2- 172.21.15.46:0/1837482722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd967160 0x55e4bd9610a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 -- 172.21.15.46:0/1837482722 >> 172.21.15.46:0/1837482722 conn(0x55e4bd9626d0 msgr2=0x55e4bd971800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 -- 172.21.15.46:0/1837482722 shutdown_connections 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 -- 172.21.15.46:0/1837482722 wait complete. 2024-04-17T07:59:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.988+0000 7fd14502b740 1 -- start start 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 1 Processor -- start 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd98ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd96ccc0 0x55e4bd98f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd976d70 0x55e4bd990290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1af680 con 0x55e4bd967160 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1b0af0 con 0x55e4bd976d70 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1afd50 con 0x55e4bd96ccc0 2024-04-17T07:59:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.989+0000 7fd14502b740 10 client.0 _ll_register_callbacks cb 0x7fd134010890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:54.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd976d70 0x55e4bd990290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd98ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd976d70 0x55e4bd990290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53416/0 (socket says 172.21.15.46:53416) 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13be00640 1 -- 172.21.15.46:0/2523232319 learned_addr learned my addr 172.21.15.46:0/2523232319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd96ccc0 0x55e4bd98f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13b400640 1 -- 172.21.15.46:0/2523232319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd96ccc0 msgr2=0x55e4bd98f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13b400640 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd96ccc0 0x55e4bd98f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13b400640 1 -- 172.21.15.46:0/2523232319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd976d70 msgr2=0x55e4bd990290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13b400640 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd976d70 0x55e4bd990290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13b400640 1 -- 172.21.15.46:0/2523232319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bd6a97d0 con 0x55e4bd967160 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.990+0000 7fd13aa00640 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd96ccc0 0x55e4bd98f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd13b400640 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd98ed40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55e4bd95e270 tx=0x7fd12c0eee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fb647c67178043 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd125600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd125600640 1 -- 172.21.15.46:0/2523232319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0bd700 con 0x55e4bd967160 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd125600640 1 -- 172.21.15.46:0/2523232319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd12c0c5ee0 con 0x55e4bd967160 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd12c0c1ab0 con 0x55e4bd967160 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd125600640 1 -- 172.21.15.46:0/2523232319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd12c0c60e0 con 0x55e4bd967160 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd14502b740 10 client.15612 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.991+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4bd9e3c80 con 0x55e4bd967160 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd125600640 1 -- 172.21.15.46:0/2523232319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12c0c1ab0 con 0x55e4bd967160 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd125600640 1 -- 172.21.15.46:0/2523232319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fd12c03c030 con 0x55e4bd967160 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd14502b740 10 client.15612 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd14502b740 10 client.15612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd14502b740 20 client.15612 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd14502b740 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd9e4380 0x55e4bd9e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd14502b740 4 client.15612 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4bd8d48b0 con 0x55e4bd9e4380 2024-04-17T07:59:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.992+0000 7fd13aa00640 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd9e4380 0x55e4bd9e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.993+0000 7fd13aa00640 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd9e4380 0x55e4bd9e6770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d727927d7169fdf6 server_cookie=f18e28a491e7b6f6 in_seq=0 out_seq=0 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.993+0000 7fd125600640 10 client.15612 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd125600640 1 -- 172.21.15.46:0/2523232319 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4bd8d48b0 con 0x55e4bd9e4380 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd125600640 10 client.15612 handle_command_reply: tid=0 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 client.15612 shutdown 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 msgr2=0x55e4bd98ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd98ed40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55e4bd95e270 tx=0x7fd12c0eee90 comp rx=0 tx=0).stop 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 shutdown_connections 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e4bd976d70 0x55e4bd990290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4bd96ccc0 0x55e4bd98f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55e4bd9e4380 0x55e4bd9e6770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 --2- 172.21.15.46:0/2523232319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e4bd967160 0x55e4bd98ed40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd125600640 0 client.15612 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T07:59:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 >> 172.21.15.46:0/2523232319 conn(0x55e4bd9626d0 msgr2=0x55e4bd9610a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 shutdown_connections 2024-04-17T07:59:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.994+0000 7fd14502b740 1 -- 172.21.15.46:0/2523232319 wait complete. 2024-04-17T07:59:54.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.995+0000 7fd14502b740 20 client.15612 trim_cache size 0 max 16384 2024-04-17T07:59:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.995+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1200a3b40 msgr2=0x7fd1200a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.995+0000 7fd140e00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1200a3b40 0x7fd1200a6000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd134009fd0 tx=0x7fd134013000 comp rx=0 tx=0).stop 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.995+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c074960 msgr2=0x7fd13c105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.995+0000 7fd140e00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c074960 0x7fd13c105170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd12c034a80 tx=0x7fd12c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.995+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 shutdown_connections 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.996+0000 7fd140e00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd1200a3b40 0x7fd1200a6000 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.996+0000 7fd140e00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd13c109c80 0x7fd13c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.996+0000 7fd140e00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c075f40 0x7fd13c1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.996+0000 7fd140e00640 1 --2- 172.21.15.46:0/962556066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd13c074960 0x7fd13c105170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.996+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 >> 172.21.15.46:0/962556066 conn(0x7fd13c0fd360 msgr2=0x7fd13c105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.996+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 shutdown_connections 2024-04-17T07:59:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:54.996+0000 7fd140e00640 1 -- 172.21.15.46:0/962556066 wait complete. 2024-04-17T07:59:55.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.250+0000 7f09e8200640 1 -- 172.21.15.46:0/3005195188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0105350 msgr2=0x7f09e0107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.250+0000 7f09e8200640 1 --2- 172.21.15.46:0/3005195188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0105350 0x7f09e0107740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009900 tx=0x7f09cc02bc40 comp rx=0 tx=0).stop 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.250+0000 7f09e8200640 1 -- 172.21.15.46:0/3005195188 shutdown_connections 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.250+0000 7f09e8200640 1 --2- 172.21.15.46:0/3005195188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09e0107c80 0x7f09e010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.250+0000 7f09e8200640 1 --2- 172.21.15.46:0/3005195188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0105350 0x7f09e0107740 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.250+0000 7f09e8200640 1 --2- 172.21.15.46:0/3005195188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e0102a20 0x7f09e0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.250+0000 7f09e8200640 1 -- 172.21.15.46:0/3005195188 >> 172.21.15.46:0/3005195188 conn(0x7f09e00fc430 msgr2=0x7f09e00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.251+0000 7f09e8200640 1 -- 172.21.15.46:0/3005195188 shutdown_connections 2024-04-17T07:59:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.251+0000 7f09e8200640 1 -- 172.21.15.46:0/3005195188 wait complete. 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 Processor -- start 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 -- start start 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e0102a20 0x7f09e019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09e0105350 0x7f09e019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0107c80 0x7f09e01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010dc40 con 0x7f09e0102a20 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010dac0 con 0x7f09e0105350 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.252+0000 7f09e8200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010ddc0 con 0x7f09e0107c80 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e0102a20 0x7f09e019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0107c80 0x7f09e01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0107c80 0x7f09e01a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44834/0 (socket says 172.21.15.46:44834) 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 -- 172.21.15.46:0/2572724703 learned_addr learned my addr 172.21.15.46:0/2572724703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e0102a20 0x7f09e019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56112/0 (socket says 172.21.15.46:56112) 2024-04-17T07:59:55.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e4e00640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09e0105350 0x7f09e019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 -- 172.21.15.46:0/2572724703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09e0105350 msgr2=0x7f09e019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09e0105350 0x7f09e019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 -- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e0102a20 msgr2=0x7f09e019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e0102a20 0x7f09e019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e6200640 1 -- 172.21.15.46:0/2572724703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e01a20d0 con 0x7f09e0107c80 2024-04-17T07:59:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.253+0000 7f09e4e00640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09e0105350 0x7f09e019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.254+0000 7f09e6200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0107c80 0x7f09e01a1a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f09d800efd0 tx=0x7f09d800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.254+0000 7f09de000640 1 -- 172.21.15.46:0/2572724703 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8003b70 con 0x7f09e0107c80 2024-04-17T07:59:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.254+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e01a23c0 con 0x7f09e0107c80 2024-04-17T07:59:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.254+0000 7f09de000640 1 -- 172.21.15.46:0/2572724703 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d8003d10 con 0x7f09e0107c80 2024-04-17T07:59:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.254+0000 7f09de000640 1 -- 172.21.15.46:0/2572724703 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80108d0 con 0x7f09e0107c80 2024-04-17T07:59:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.254+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01a28e0 con 0x7f09e0107c80 2024-04-17T07:59:55.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.257+0000 7f09eafd8740 1 Processor -- start 2024-04-17T07:59:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.259+0000 7f09de000640 1 -- 172.21.15.46:0/2572724703 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f09d8010a90 con 0x7f09e0107c80 2024-04-17T07:59:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.259+0000 7f09eafd8740 1 -- start start 2024-04-17T07:59:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2bef33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2bef8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2befaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09eafd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55af2be59020 con 0x55af2bef2fa0 2024-04-17T07:59:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09eafd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af2bdff460 con 0x55af2bef8b00 2024-04-17T07:59:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09eafd8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af2be54fe0 con 0x55af2bef3990 2024-04-17T07:59:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09e5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2bef8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09e4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2bef33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09e5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2bef8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44836/0 (socket says 172.21.15.46:44836) 2024-04-17T07:59:55.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09e5800640 1 -- 172.21.15.46:0/799746507 learned_addr learned my addr 172.21.15.46:0/799746507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:55.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.260+0000 7f09e4e00640 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2befaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09e5800640 1 -- 172.21.15.46:0/799746507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 msgr2=0x55af2befaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:55.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09de000640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09c80a3c30 0x7f09c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09e5800640 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2befaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09e5800640 1 -- 172.21.15.46:0/799746507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 msgr2=0x55af2bef33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09e5800640 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2bef33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09e5800640 1 -- 172.21.15.46:0/799746507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2be07040 con 0x55af2bef3990 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09e4e00640 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2bef33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09de000640 1 -- 172.21.15.46:0/2572724703 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f09d80d0b50 con 0x7f09e0107c80 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.261+0000 7f09e5800640 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2bef8490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09d400ad00 tx=0x7f09d400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71adbec444d96f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.262+0000 7f09e6200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09c80a3c30 0x7f09c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.263+0000 7f09d2a00640 1 -- 172.21.15.46:0/799746507 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d401d070 con 0x55af2bef3990 2024-04-17T07:59:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.263+0000 7f09d2a00640 1 -- 172.21.15.46:0/799746507 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4018b20 con 0x55af2bef3990 2024-04-17T07:59:55.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.263+0000 7f09d2a00640 1 -- 172.21.15.46:0/799746507 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4010c40 con 0x55af2bef3990 2024-04-17T07:59:55.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.263+0000 7f09eafd8740 1 -- 172.21.15.46:0/799746507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 msgr2=0x55af2bef8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.263+0000 7f09eafd8740 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2bef8490 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09d400ad00 tx=0x7f09d400f790 comp rx=0 tx=0).stop 2024-04-17T07:59:55.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.263+0000 7f09e6200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09c80a3c30 0x7f09c80a60f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005c40 tx=0x7f09d8004900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:55.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 -- 172.21.15.46:0/799746507 shutdown_connections 2024-04-17T07:59:55.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2befaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2bef8490 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 --2- 172.21.15.46:0/799746507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2bef33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 -- 172.21.15.46:0/799746507 >> 172.21.15.46:0/799746507 conn(0x55af2beee510 msgr2=0x55af2beee910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:55.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 -- 172.21.15.46:0/799746507 shutdown_connections 2024-04-17T07:59:55.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 -- 172.21.15.46:0/799746507 wait complete. 2024-04-17T07:59:55.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 -- start start 2024-04-17T07:59:55.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.264+0000 7f09eafd8740 1 Processor -- start 2024-04-17T07:59:55.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2befd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2befda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2bf04ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09eafd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55af2befbab0 con 0x55af2bef2fa0 2024-04-17T07:59:55.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09eafd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af2be12830 con 0x55af2bef8b00 2024-04-17T07:59:55.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09eafd8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af2befbc30 con 0x55af2bef3990 2024-04-17T07:59:55.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09eafd8740 10 client.0 _ll_register_callbacks cb 0x55af2bde0fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:55.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09e5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2befda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09e4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2befd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09e5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2befda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44842/0 (socket says 172.21.15.46:44842) 2024-04-17T07:59:55.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.265+0000 7f09e5800640 1 -- 172.21.15.46:0/3662921485 learned_addr learned my addr 172.21.15.46:0/3662921485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.266+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2bf04ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.266+0000 7f09e5800640 1 -- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 msgr2=0x55af2bf04ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.266+0000 7f09e5800640 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2bf04ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.266+0000 7f09e5800640 1 -- 172.21.15.46:0/3662921485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 msgr2=0x55af2befd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.266+0000 7f09e5800640 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2befd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.266+0000 7f09e5800640 1 -- 172.21.15.46:0/3662921485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2be16920 con 0x55af2bef3990 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09e5800640 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2befda80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000c00 tx=0x7f09d4011b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c099a5804c7668 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09c7400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09c7400640 1 -- 172.21.15.46:0/3662921485 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d401d050 con 0x55af2bef3990 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d4018b20 con 0x55af2bef3990 2024-04-17T07:59:55.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09c7400640 1 -- 172.21.15.46:0/3662921485 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f09d4018640 con 0x55af2bef3990 2024-04-17T07:59:55.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09c7400640 1 -- 172.21.15.46:0/3662921485 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f09d4032050 con 0x55af2bef3990 2024-04-17T07:59:55.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09eafd8740 10 client.25750 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.267+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55af2bde47f0 con 0x55af2bef3990 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09c7400640 1 -- 172.21.15.46:0/3662921485 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4018b20 con 0x55af2bef3990 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09c7400640 1 -- 172.21.15.46:0/3662921485 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f09d4004960 con 0x55af2bef3990 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09eafd8740 10 client.25750 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09eafd8740 10 client.25750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09eafd8740 20 client.25750 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf08c20 0x55af2bf0b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09eafd8740 4 client.25750 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.268+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af2be66a40 con 0x55af2bf08c20 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.269+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf08c20 0x55af2bf0b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.269+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf08c20 0x55af2bf0b010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0c87e2a32bc52 server_cookie=2e444bff52a2691d in_seq=0 out_seq=0 2024-04-17T07:59:55.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.269+0000 7f09c7400640 10 client.25750 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:55.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.270+0000 7f09c7400640 1 -- 172.21.15.46:0/3662921485 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af2be66a40 con 0x55af2bf08c20 2024-04-17T07:59:55.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.270+0000 7f09c7400640 10 client.25750 handle_command_reply: tid=0 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.270+0000 7f09eafd8740 1 client.25750 shutdown 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 msgr2=0x55af2befda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2befda80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09d4000c00 tx=0x7f09d4011b80 comp rx=0 tx=0).stop 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 shutdown_connections 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf08c20 0x55af2bf0b010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef8b00 0x55af2bf04ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef3990 0x55af2befda80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3662921485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef2fa0 0x55af2befd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09c7400640 0 client.25750 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 >> 172.21.15.46:0/3662921485 conn(0x55af2beee510 msgr2=0x55af2beed710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 shutdown_connections 2024-04-17T07:59:55.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 1 -- 172.21.15.46:0/3662921485 wait complete. 2024-04-17T07:59:55.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.271+0000 7f09eafd8740 20 client.25750 trim_cache size 0 max 16384 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 Processor -- start 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 -- start start 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2befbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2befc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf04a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01af6c0 con 0x55af2bef8b00 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01acf80 con 0x55af2bef3990 2024-04-17T07:59:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09eafd8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01aadd0 con 0x55af2bef2fa0 2024-04-17T07:59:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09e5800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2befc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09e5800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2befc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53470/0 (socket says 172.21.15.46:53470) 2024-04-17T07:59:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.286+0000 7f09e5800640 1 -- 172.21.15.46:0/3300416136 learned_addr learned my addr 172.21.15.46:0/3300416136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e5800640 1 -- 172.21.15.46:0/3300416136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 msgr2=0x55af2befbfd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:59:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e5800640 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2befbfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e5800640 1 -- 172.21.15.46:0/3300416136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 msgr2=0x55af2bf04a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:59:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf04a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e5800640 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf04a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e5800640 1 -- 172.21.15.46:0/3300416136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2bb6b3f0 con 0x55af2bef3990 2024-04-17T07:59:55.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf04a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:59:55.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.287+0000 7f09e5800640 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2befc510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f09d4011920 tx=0x7f09d400ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31bedd678c78acbe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.288+0000 7f09d2000640 1 -- 172.21.15.46:0/3300416136 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d401d040 con 0x55af2bef3990 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.288+0000 7f09d2000640 1 -- 172.21.15.46:0/3300416136 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4029070 con 0x55af2bef3990 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.288+0000 7f09d2000640 1 -- 172.21.15.46:0/3300416136 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4005360 con 0x55af2bef3990 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.288+0000 7f09eafd8740 1 -- 172.21.15.46:0/3300416136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 msgr2=0x55af2befc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.288+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2befc510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f09d4011920 tx=0x7f09d400ac00 comp rx=0 tx=0).stop 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 -- 172.21.15.46:0/3300416136 shutdown_connections 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf04a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2befc510 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3300416136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2befbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 -- 172.21.15.46:0/3300416136 >> 172.21.15.46:0/3300416136 conn(0x55af2beee510 msgr2=0x55af2bef4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 -- 172.21.15.46:0/3300416136 shutdown_connections 2024-04-17T07:59:55.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 -- 172.21.15.46:0/3300416136 wait complete. 2024-04-17T07:59:55.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 -- start start 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.289+0000 7f09eafd8740 1 Processor -- start 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2bf01750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2bf01c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09eafd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09eafd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01ab440 con 0x55af2bef8b00 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09eafd8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01ac8b0 con 0x55af2bef3990 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09eafd8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01abb10 con 0x55af2bef2fa0 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09eafd8740 10 client.0 _ll_register_callbacks cb 0x7f09d400a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09e4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2bf01750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09e4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2bf01750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44860/0 (socket says 172.21.15.46:44860) 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09e4e00640 1 -- 172.21.15.46:0/3287216244 learned_addr learned my addr 172.21.15.46:0/3287216244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.290+0000 7f09e5800640 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2bf01c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.291+0000 7f09e5800640 1 -- 172.21.15.46:0/3287216244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 msgr2=0x55af2bf01750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.291+0000 7f09e5800640 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2bf01750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.291+0000 7f09e5800640 1 -- 172.21.15.46:0/3287216244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 msgr2=0x55af2bf6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.291+0000 7f09e5800640 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.291+0000 7f09e5800640 1 -- 172.21.15.46:0/3287216244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2bc357d0 con 0x55af2bef3990 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.291+0000 7f09e5800640 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2bf01c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09d4004e80 tx=0x7f09d40245d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c681a46fb8c5980 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.291+0000 7f09c7400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:59:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.292+0000 7f09c7400640 1 -- 172.21.15.46:0/3287216244 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4042020 con 0x55af2bef3990 2024-04-17T07:59:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.292+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d4029070 con 0x55af2bef3990 2024-04-17T07:59:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.292+0000 7f09c7400640 1 -- 172.21.15.46:0/3287216244 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f09d4004a70 con 0x55af2bef3990 2024-04-17T07:59:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.292+0000 7f09c7400640 1 -- 172.21.15.46:0/3287216244 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f09d404a050 con 0x55af2bef3990 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.292+0000 7f09eafd8740 10 client.25769 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.292+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55af2bf6e4b0 con 0x55af2bef3990 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.292+0000 7f09c7400640 1 -- 172.21.15.46:0/3287216244 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4029070 con 0x55af2bef3990 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09c7400640 1 -- 172.21.15.46:0/3287216244 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f09d40114f0 con 0x55af2bef3990 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09eafd8740 10 client.25769 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09eafd8740 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09eafd8740 20 client.25769 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf6ee50 0x55af2bf71270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09eafd8740 4 client.25769 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af2be608b0 con 0x55af2bf6ee50 2024-04-17T07:59:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.293+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf6ee50 0x55af2bf71270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09e4e00640 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf6ee50 0x55af2bf71270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec03a8ce98743b6b server_cookie=8245df87e99744d6 in_seq=0 out_seq=0 2024-04-17T07:59:55.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09c7400640 10 client.25769 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09c7400640 1 -- 172.21.15.46:0/3287216244 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af2be608b0 con 0x55af2bf6ee50 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09c7400640 10 client.25769 handle_command_reply: tid=0 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 client.25769 shutdown 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 msgr2=0x55af2bf01c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2bf01c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09d4004e80 tx=0x7f09d40245d0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 shutdown_connections 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55af2bf6ee50 0x55af2bf71270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af2bef8b00 0x55af2bf6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af2bef3990 0x55af2bf01c90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 --2- 172.21.15.46:0/3287216244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af2bef2fa0 0x55af2bf01750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.294+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 >> 172.21.15.46:0/3287216244 conn(0x55af2beee510 msgr2=0x55af2befd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.295+0000 7f09c7400640 0 client.25769 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.295+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 shutdown_connections 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.295+0000 7f09eafd8740 1 -- 172.21.15.46:0/3287216244 wait complete. 2024-04-17T07:59:55.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.295+0000 7f09eafd8740 20 client.25769 trim_cache size 0 max 16384 2024-04-17T07:59:55.296 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T07:59:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09c80a3c30 msgr2=0x7f09c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09c80a3c30 0x7f09c80a60f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005c40 tx=0x7f09d8004900 comp rx=0 tx=0).stop 2024-04-17T07:59:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0107c80 msgr2=0x7f09e01a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0107c80 0x7f09e01a1a60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f09d800efd0 tx=0x7f09d800c6a0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 shutdown_connections 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f09c80a3c30 0x7f09c80a60f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09e0107c80 0x7f09e01a1a60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09e0105350 0x7f09e019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 --2- 172.21.15.46:0/2572724703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09e0102a20 0x7f09e019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 >> 172.21.15.46:0/2572724703 conn(0x7f09e00fc430 msgr2=0x7f09e01020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 shutdown_connections 2024-04-17T07:59:55.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:55.296+0000 7f09e8200640 1 -- 172.21.15.46:0/2572724703 wait complete. 2024-04-17T07:59:55.316 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:55 smithi046 ceph-mon[32038]: pgmap v459: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:55.316 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:55 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3261667579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:59:55.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:55 smithi062 ceph-mon[36507]: pgmap v459: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:55.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:55 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3261667579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:59:55.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:55 smithi053 ceph-mon[36437]: pgmap v459: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T07:59:55.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:55 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3261667579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:59:56.321 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:59:56.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:59:56.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:59:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.616+0000 7f0c8dc00640 1 -- 172.21.15.46:0/2366489337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88075f20 msgr2=0x7f0c88074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.616+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/2366489337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88075f20 0x7f0c88074370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009900 tx=0x7f0c7802bcc0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.617+0000 7f0c8dc00640 1 -- 172.21.15.46:0/2366489337 shutdown_connections 2024-04-17T07:59:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.617+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/2366489337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88109ca0 0x7f0c8810c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.617+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/2366489337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c88074940 0x7f0c88074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.617+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/2366489337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88075f20 0x7f0c88074370 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.617+0000 7f0c8dc00640 1 -- 172.21.15.46:0/2366489337 >> 172.21.15.46:0/2366489337 conn(0x7f0c880fd360 msgr2=0x7f0c880ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.617+0000 7f0c8dc00640 1 -- 172.21.15.46:0/2366489337 shutdown_connections 2024-04-17T07:59:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.617+0000 7f0c8dc00640 1 -- 172.21.15.46:0/2366489337 wait complete. 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 Processor -- start 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 -- start start 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88074940 0x7f0c8819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c88075f20 0x7f0c8819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88109ca0 0x7f0c881a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8810fce0 con 0x7f0c88109ca0 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8810fb60 con 0x7f0c88074940 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.618+0000 7f0c8dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8810fe60 con 0x7f0c88075f20 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c8d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88109ca0 0x7f0c881a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88074940 0x7f0c8819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c88075f20 0x7f0c8819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88074940 0x7f0c8819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53510/0 (socket says 172.21.15.46:53510) 2024-04-17T07:59:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 -- 172.21.15.46:0/1381555738 learned_addr learned my addr 172.21.15.46:0/1381555738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 -- 172.21.15.46:0/1381555738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c88075f20 msgr2=0x7f0c8819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c88075f20 0x7f0c8819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 -- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88109ca0 msgr2=0x7f0c881a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88109ca0 0x7f0c881a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.619+0000 7f0c87e00640 1 -- 172.21.15.46:0/1381555738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c881a1f10 con 0x7f0c88074940 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.620+0000 7f0c87e00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88074940 0x7f0c8819a2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c78000c00 tx=0x7f0c78005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.620+0000 7f0c84c00640 1 -- 172.21.15.46:0/1381555738 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78043020 con 0x7f0c88074940 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.620+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c881a21a0 con 0x7f0c88074940 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.620+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c881a2740 con 0x7f0c88074940 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.620+0000 7f0c84c00640 1 -- 172.21.15.46:0/1381555738 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c780054e0 con 0x7f0c88074940 2024-04-17T07:59:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.620+0000 7f0c84c00640 1 -- 172.21.15.46:0/1381555738 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78003d20 con 0x7f0c88074940 2024-04-17T07:59:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.623+0000 7f0c84c00640 1 -- 172.21.15.46:0/1381555738 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0c78046020 con 0x7f0c88074940 2024-04-17T07:59:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.624+0000 7f0c84c00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0c6c0a3c30 0x7f0c6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.624+0000 7f0c91f68740 1 Processor -- start 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c84c00640 1 -- 172.21.15.46:0/1381555738 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f0c780f2860 con 0x7f0c88074940 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c91f68740 1 -- start start 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c87400640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0c6c0a3c30 0x7f0c6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df030a0 0x56424df034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df08590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c91f68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56424de68cb0 con 0x56424df03a90 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c91f68740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56424de0f370 con 0x56424df030a0 2024-04-17T07:59:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c91f68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56424de64c70 con 0x56424df08c00 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.625+0000 7f0c87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df08590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df08590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56182/0 (socket says 172.21.15.46:56182) 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c87400640 1 -- 172.21.15.46:0/2460317715 learned_addr learned my addr 172.21.15.46:0/2460317715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c87e00640 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df030a0 0x56424df034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c8d200640 1 -- 172.21.15.46:0/2460317715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df030a0 msgr2=0x56424df034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df030a0 0x56424df034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c8d200640 1 -- 172.21.15.46:0/2460317715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 msgr2=0x56424df08590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df08590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c8d200640 1 -- 172.21.15.46:0/2460317715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424de2e910 con 0x56424df08c00 2024-04-17T07:59:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c87e00640 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df030a0 0x56424df034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df0b090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009f30 tx=0x7f0c80010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8296e7a3f82a32e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.626+0000 7f0c7d600640 1 -- 172.21.15.46:0/2460317715 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80016070 con 0x56424df08c00 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.627+0000 7f0c7d600640 1 -- 172.21.15.46:0/2460317715 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c80004d20 con 0x56424df08c00 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.627+0000 7f0c7d600640 1 -- 172.21.15.46:0/2460317715 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80012810 con 0x56424df08c00 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.627+0000 7f0c91f68740 1 -- 172.21.15.46:0/2460317715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 msgr2=0x56424df0b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.627+0000 7f0c87400640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0c6c0a3c30 0x7f0c6c0a60f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c74012620 tx=0x7f0c7400bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.627+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df0b090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009f30 tx=0x7f0c80010800 comp rx=0 tx=0).stop 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 -- 172.21.15.46:0/2460317715 shutdown_connections 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df0b090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df08590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2460317715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df030a0 0x56424df034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 -- 172.21.15.46:0/2460317715 >> 172.21.15.46:0/2460317715 conn(0x56424defe610 msgr2=0x56424defea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 -- 172.21.15.46:0/2460317715 shutdown_connections 2024-04-17T07:59:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 -- 172.21.15.46:0/2460317715 wait complete. 2024-04-17T07:59:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.628+0000 7f0c91f68740 1 -- start start 2024-04-17T07:59:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 1 Processor -- start 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df030a0 0x56424df0d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df0dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424df14b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56424df0bbb0 con 0x56424df03a90 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56424de22730 con 0x56424df08c00 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56424df0bd30 con 0x56424df030a0 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.629+0000 7f0c91f68740 10 client.0 _ll_register_callbacks cb 0x56424de24b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df030a0 0x56424df0d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df030a0 0x56424df0d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44910/0 (socket says 172.21.15.46:44910) 2024-04-17T07:59:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 -- 172.21.15.46:0/2826799468 learned_addr learned my addr 172.21.15.46:0/2826799468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424df14b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87400640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df0dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 -- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 msgr2=0x56424df14b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424df14b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 -- 172.21.15.46:0/2826799468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 msgr2=0x56424df0dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df0dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 -- 172.21.15.46:0/2826799468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424dc437e0 con 0x56424df030a0 2024-04-17T07:59:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87400640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df0dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c87e00640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df030a0 0x56424df0d590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0c780effd0 tx=0x7f0c78005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edace3a3ac7a3db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.630+0000 7f0c72000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.631+0000 7f0c72000640 1 -- 172.21.15.46:0/2826799468 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c780beeb0 con 0x56424df030a0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.631+0000 7f0c72000640 1 -- 172.21.15.46:0/2826799468 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f0c780a90b0 con 0x56424df030a0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.631+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c80004d20 con 0x56424df030a0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.631+0000 7f0c72000640 1 -- 172.21.15.46:0/2826799468 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c780be7b0 con 0x56424df030a0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.631+0000 7f0c91f68740 10 client.25765 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.631+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56424dca3f70 con 0x56424df030a0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.631+0000 7f0c72000640 1 -- 172.21.15.46:0/2826799468 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c80004d20 con 0x56424df030a0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c72000640 1 -- 172.21.15.46:0/2826799468 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f0c780c6f90 con 0x56424df030a0 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c91f68740 10 client.25765 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c91f68740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c91f68740 20 client.25765 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df19480 0x56424df1b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c91f68740 4 client.25765 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56424de766d0 con 0x56424df19480 2024-04-17T07:59:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.632+0000 7f0c87400640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df19480 0x56424df1b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.633+0000 7f0c87400640 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df19480 0x56424df1b910 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a156eab81f25f0 server_cookie=6e383849210f5967 in_seq=0 out_seq=0 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.633+0000 7f0c72000640 10 client.25765 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c72000640 1 -- 172.21.15.46:0/2826799468 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56424de766d0 con 0x56424df19480 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c72000640 10 client.25765 handle_command_reply: tid=0 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 client.25765 shutdown 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df030a0 msgr2=0x56424df0d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df030a0 0x56424df0d590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0c780effd0 tx=0x7f0c78005fd0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 shutdown_connections 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df19480 0x56424df1b910 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424df14b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df03a90 0x56424df0dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 --2- 172.21.15.46:0/2826799468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df030a0 0x56424df0d590 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c72000640 0 client.25765 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 >> 172.21.15.46:0/2826799468 conn(0x56424defe610 msgr2=0x56424defd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 shutdown_connections 2024-04-17T07:59:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 1 -- 172.21.15.46:0/2826799468 wait complete. 2024-04-17T07:59:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.634+0000 7f0c91f68740 20 client.25765 trim_cache size 0 max 16384 2024-04-17T07:59:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.653+0000 7f0c91f68740 1 Processor -- start 2024-04-17T07:59:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.653+0000 7f0c91f68740 1 -- start start 2024-04-17T07:59:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.653+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424def9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df03a90 0x56424df0d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424defd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c91f68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881ab050 con 0x56424df030a0 2024-04-17T07:59:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c91f68740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881a8910 con 0x56424df08c00 2024-04-17T07:59:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c91f68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881a6760 con 0x56424df03a90 2024-04-17T07:59:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424def9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df03a90 0x56424df0d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df03a90 0x56424df0d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44918/0 (socket says 172.21.15.46:44918) 2024-04-17T07:59:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 -- 172.21.15.46:0/1142617107 learned_addr learned my addr 172.21.15.46:0/1142617107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c8d200640 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424defd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 -- 172.21.15.46:0/1142617107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 msgr2=0x56424defd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424defd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 -- 172.21.15.46:0/1142617107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 msgr2=0x56424def9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424def9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.654+0000 7f0c87400640 1 -- 172.21.15.46:0/1142617107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424db3b170 con 0x56424df03a90 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.655+0000 7f0c87400640 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df03a90 0x56424df0d8b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c74005f00 tx=0x7f0c7401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4844f702ae56fef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.655+0000 7f0c7cc00640 1 -- 172.21.15.46:0/1142617107 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74021110 con 0x56424df03a90 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.655+0000 7f0c7cc00640 1 -- 172.21.15.46:0/1142617107 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7400be40 con 0x56424df03a90 2024-04-17T07:59:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.655+0000 7f0c7cc00640 1 -- 172.21.15.46:0/1142617107 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74028e20 con 0x56424df03a90 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 -- 172.21.15.46:0/1142617107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df03a90 msgr2=0x56424df0d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df03a90 0x56424df0d8b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c74005f00 tx=0x7f0c7401c000 comp rx=0 tx=0).stop 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 -- 172.21.15.46:0/1142617107 shutdown_connections 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df08c00 0x56424defd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df03a90 0x56424df0d8b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 --2- 172.21.15.46:0/1142617107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424def9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 -- 172.21.15.46:0/1142617107 >> 172.21.15.46:0/1142617107 conn(0x56424defe610 msgr2=0x56424df0d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 -- 172.21.15.46:0/1142617107 shutdown_connections 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.656+0000 7f0c91f68740 1 -- 172.21.15.46:0/1142617107 wait complete. 2024-04-17T07:59:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.657+0000 7f0c91f68740 1 -- start start 2024-04-17T07:59:56.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.657+0000 7f0c91f68740 1 Processor -- start 2024-04-17T07:59:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424df13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df03a90 0x56424df13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c91f68740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df11ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c91f68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881a6dd0 con 0x56424df030a0 2024-04-17T07:59:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c91f68740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881a8240 con 0x56424df03a90 2024-04-17T07:59:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c91f68740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0c881a74a0 con 0x56424df08c00 2024-04-17T07:59:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c91f68740 10 client.0 _ll_register_callbacks cb 0x7f0c780095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424df13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424df13840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56206/0 (socket says 172.21.15.46:56206) 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 -- 172.21.15.46:0/396924132 learned_addr learned my addr 172.21.15.46:0/396924132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87400640 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df03a90 0x56424df13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c8d200640 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df11ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 -- 172.21.15.46:0/396924132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 msgr2=0x56424df11ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df11ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 -- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df03a90 msgr2=0x56424df13d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df03a90 0x56424df13d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.658+0000 7f0c87e00640 1 -- 172.21.15.46:0/396924132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424df12530 con 0x56424df030a0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c87e00640 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424df13840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c780c7330 tx=0x7f0c78003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1673610287feaae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c72000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c72000640 1 -- 172.21.15.46:0/396924132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c780beeb0 con 0x56424df030a0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c72000640 1 -- 172.21.15.46:0/396924132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f0c780bdc00 con 0x56424df030a0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7400be40 con 0x56424df030a0 2024-04-17T07:59:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c72000640 1 -- 172.21.15.46:0/396924132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0c780ae050 con 0x56424df030a0 2024-04-17T07:59:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c91f68740 10 client.15639 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.659+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56424df80410 con 0x56424df030a0 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c72000640 1 -- 172.21.15.46:0/396924132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7400be40 con 0x56424df030a0 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c72000640 1 -- 172.21.15.46:0/396924132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f0c780ba620 con 0x56424df030a0 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c91f68740 10 client.15639 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c91f68740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c91f68740 20 client.15639 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c91f68740 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df809e0 0x56424df82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c91f68740 4 client.15639 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.660+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56424de70540 con 0x56424df809e0 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.661+0000 7f0c87400640 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df809e0 0x56424df82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.661+0000 7f0c72000640 10 client.15639 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.661+0000 7f0c87400640 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df809e0 0x56424df82e00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3265edd0bda2be server_cookie=54bb456cd2aa3605 in_seq=0 out_seq=0 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.662+0000 7f0c72000640 1 -- 172.21.15.46:0/396924132 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56424de70540 con 0x56424df809e0 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.662+0000 7f0c72000640 10 client.15639 handle_command_reply: tid=0 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.662+0000 7f0c91f68740 1 client.15639 shutdown 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 msgr2=0x56424df13840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424df13840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0c780c7330 tx=0x7f0c78003820 comp rx=0 tx=0).stop 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 shutdown_connections 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56424df809e0 0x56424df82e00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56424df08c00 0x56424df11ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56424df03a90 0x56424df13d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 --2- 172.21.15.46:0/396924132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424df030a0 0x56424df13840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 >> 172.21.15.46:0/396924132 conn(0x56424defe610 msgr2=0x56424df01870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c72000640 0 client.15639 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 shutdown_connections 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.663+0000 7f0c91f68740 1 -- 172.21.15.46:0/396924132 wait complete. 2024-04-17T07:59:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.664+0000 7f0c91f68740 20 client.15639 trim_cache size 0 max 16384 2024-04-17T07:59:56.664 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:59:56.664 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-17T07:59:56.664 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a41451d0-4136-403e-b5b9-d06a5c76c91e", 2024-04-17T07:59:56.664 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-17T07:59:56.664 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.665+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0c6c0a3c30 msgr2=0x7f0c6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.665+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0c6c0a3c30 0x7f0c6c0a60f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c74012620 tx=0x7f0c7400bb70 comp rx=0 tx=0).stop 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.665+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88074940 msgr2=0x7f0c8819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.665+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88074940 0x7f0c8819a2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c78000c00 tx=0x7f0c78005dd0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 shutdown_connections 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f0c6c0a3c30 0x7f0c6c0a60f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88109ca0 0x7f0c881a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c88075f20 0x7f0c8819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 --2- 172.21.15.46:0/1381555738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c88074940 0x7f0c8819a2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 >> 172.21.15.46:0/1381555738 conn(0x7f0c880fd360 msgr2=0x7f0c88105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 shutdown_connections 2024-04-17T07:59:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.666+0000 7f0c8dc00640 1 -- 172.21.15.46:0/1381555738 wait complete. 2024-04-17T07:59:56.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:59:56.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:59:56.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 -- 172.21.15.46:0/3386509023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 msgr2=0x7f840c10c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 --2- 172.21.15.46:0/3386509023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 0x7f840c10c130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8404009be0 tx=0x7f840402bf80 comp rx=0 tx=0).stop 2024-04-17T07:59:56.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 -- 172.21.15.46:0/3386509023 shutdown_connections 2024-04-17T07:59:56.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 --2- 172.21.15.46:0/3386509023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 0x7f840c10c130 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 --2- 172.21.15.46:0/3386509023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c074940 0x7f840c074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 --2- 172.21.15.46:0/3386509023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 0x7f840c074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 -- 172.21.15.46:0/3386509023 >> 172.21.15.46:0/3386509023 conn(0x7f840c0fd360 msgr2=0x7f840c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 -- 172.21.15.46:0/3386509023 shutdown_connections 2024-04-17T07:59:56.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.869+0000 7f8412600640 1 -- 172.21.15.46:0/3386509023 wait complete. 2024-04-17T07:59:56.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 Processor -- start 2024-04-17T07:59:56.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 -- start start 2024-04-17T07:59:56.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c074940 0x7f840c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 0x7f840c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 0x7f840c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f840c10fc90 con 0x7f840c075f20 2024-04-17T07:59:56.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f840c10fb10 con 0x7f840c074940 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.871+0000 7f8412600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f840c10fe10 con 0x7f840c109ca0 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f840be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 0x7f840c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f840be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 0x7f840c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56230/0 (socket says 172.21.15.46:56230) 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f8411c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 0x7f840c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f840be00640 1 -- 172.21.15.46:0/801093709 learned_addr learned my addr 172.21.15.46:0/801093709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f8411200640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c074940 0x7f840c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f8411c00640 1 -- 172.21.15.46:0/801093709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c074940 msgr2=0x7f840c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f8411c00640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c074940 0x7f840c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f8411c00640 1 -- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 msgr2=0x7f840c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f8411c00640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 0x7f840c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f8411c00640 1 -- 172.21.15.46:0/801093709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c1a1e60 con 0x7f840c109ca0 2024-04-17T07:59:56.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.872+0000 7f840be00640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 0x7f840c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:59:56.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.873+0000 7f8411c00640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 0x7f840c1a17f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8404036040 tx=0x7f8404004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.873+0000 7f8409600640 1 -- 172.21.15.46:0/801093709 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404043020 con 0x7f840c109ca0 2024-04-17T07:59:56.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.873+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c1a20f0 con 0x7f840c109ca0 2024-04-17T07:59:56.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.873+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c1a2690 con 0x7f840c109ca0 2024-04-17T07:59:56.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.873+0000 7f8409600640 1 -- 172.21.15.46:0/801093709 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8404004670 con 0x7f840c109ca0 2024-04-17T07:59:56.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.873+0000 7f8409600640 1 -- 172.21.15.46:0/801093709 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84040347b0 con 0x7f840c109ca0 2024-04-17T07:59:56.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.877+0000 7f8416a2e740 1 Processor -- start 2024-04-17T07:59:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.878+0000 7f8409600640 1 -- 172.21.15.46:0/801093709 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8404046020 con 0x7f840c109ca0 2024-04-17T07:59:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.878+0000 7f8416a2e740 1 -- start start 2024-04-17T07:59:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e483524fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f8416a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e483483120 con 0x55e483524b80 2024-04-17T07:59:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f8416a2e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e483429340 con 0x55e483525570 2024-04-17T07:59:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f8416a2e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e48347f0e0 con 0x55e48352a6e0 2024-04-17T07:59:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f840be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e483524fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f840be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e483524fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56240/0 (socket says 172.21.15.46:56240) 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f840be00640 1 -- 172.21.15.46:0/830820920 learned_addr learned my addr 172.21.15.46:0/830820920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.879+0000 7f8411200640 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f840be00640 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f8411200640 1 -- 172.21.15.46:0/830820920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 msgr2=0x55e48352cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f8411200640 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f8411200640 1 -- 172.21.15.46:0/830820920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 msgr2=0x55e483524fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f8411200640 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e483524fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f8411200640 1 -- 172.21.15.46:0/830820920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e48325f7d0 con 0x55e483525570 2024-04-17T07:59:56.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f8411200640 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352a070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f840000bbf0 tx=0x7f84000107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6ef5a2dfc8491 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.880+0000 7f8409600640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83f40a3c30 0x7f83f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.881+0000 7f8409600640 1 -- 172.21.15.46:0/801093709 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f84040f2d90 con 0x7f840c109ca0 2024-04-17T07:59:56.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.881+0000 7f83fe000640 1 -- 172.21.15.46:0/830820920 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400004030 con 0x55e483525570 2024-04-17T07:59:56.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.881+0000 7f83fe000640 1 -- 172.21.15.46:0/830820920 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84000041d0 con 0x55e483525570 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.881+0000 7f83fe000640 1 -- 172.21.15.46:0/830820920 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84000116c0 con 0x55e483525570 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.881+0000 7f840be00640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83f40a3c30 0x7f83f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.881+0000 7f8416a2e740 1 -- 172.21.15.46:0/830820920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 msgr2=0x55e48352a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.881+0000 7f8416a2e740 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352a070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f840000bbf0 tx=0x7f84000107a0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.882+0000 7f8416a2e740 1 -- 172.21.15.46:0/830820920 shutdown_connections 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.882+0000 7f8416a2e740 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.882+0000 7f8416a2e740 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352a070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.882+0000 7f8416a2e740 1 --2- 172.21.15.46:0/830820920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e483524fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.882+0000 7f8416a2e740 1 -- 172.21.15.46:0/830820920 >> 172.21.15.46:0/830820920 conn(0x55e4835178e0 msgr2=0x55e483517ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.882+0000 7f840be00640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83f40a3c30 0x7f83f40a60f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f83f80096c0 tx=0x7f83f8005a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.882+0000 7f8416a2e740 1 -- 172.21.15.46:0/830820920 shutdown_connections 2024-04-17T07:59:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.883+0000 7f8416a2e740 1 -- 172.21.15.46:0/830820920 wait complete. 2024-04-17T07:59:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.883+0000 7f8416a2e740 1 -- start start 2024-04-17T07:59:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.883+0000 7f8416a2e740 1 Processor -- start 2024-04-17T07:59:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e483524b80 0x55e48352ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e48352a6e0 0x55e483536410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8416a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e48352d690 con 0x55e48352a6e0 2024-04-17T07:59:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8416a2e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e48343c710 con 0x55e483525570 2024-04-17T07:59:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8416a2e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e48352d810 con 0x55e483524b80 2024-04-17T07:59:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8416a2e740 10 client.0 _ll_register_callbacks cb 0x55e483437af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e483524b80 0x55e48352ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e483524b80 0x55e48352ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44974/0 (socket says 172.21.15.46:44974) 2024-04-17T07:59:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 -- 172.21.15.46:0/402371012 learned_addr learned my addr 172.21.15.46:0/402371012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f840be00640 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411c00640 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e48352a6e0 0x55e483536410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 -- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 msgr2=0x55e48352f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 -- 172.21.15.46:0/402371012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e48352a6e0 msgr2=0x55e483536410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e48352a6e0 0x55e483536410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.884+0000 7f8411200640 1 -- 172.21.15.46:0/402371012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4831953f0 con 0x55e483524b80 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f8411200640 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e483524b80 0x55e48352ee90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55e483525fe0 tx=0x7f840000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1fe5f5d3e577f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f83f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f83f2a00640 1 -- 172.21.15.46:0/402371012 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400003a00 con 0x55e483524b80 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f83f2a00640 1 -- 172.21.15.46:0/402371012 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8400005040 con 0x55e483524b80 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84000041d0 con 0x55e483524b80 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f83f2a00640 1 -- 172.21.15.46:0/402371012 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8400032050 con 0x55e483524b80 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f8416a2e740 10 client.25783 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.885+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e4831550f0 con 0x55e483524b80 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.886+0000 7f83f2a00640 1 -- 172.21.15.46:0/402371012 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84000041d0 con 0x55e483524b80 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.886+0000 7f83f2a00640 1 -- 172.21.15.46:0/402371012 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f84000052c0 con 0x55e483524b80 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.887+0000 7f8416a2e740 10 client.25783 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.887+0000 7f8416a2e740 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.887+0000 7f8416a2e740 20 client.25783 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.887+0000 7f8416a2e740 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e48353a900 0x55e48353ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.887+0000 7f8416a2e740 4 client.25783 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:59:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.887+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e483490b40 con 0x55e48353a900 2024-04-17T07:59:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.887+0000 7f840be00640 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e48353a900 0x55e48353ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.888+0000 7f840be00640 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e48353a900 0x55e48353ccf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af9d8822603ba8c server_cookie=591f0168a0b0855e in_seq=0 out_seq=0 2024-04-17T07:59:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.888+0000 7f83f2a00640 10 client.25783 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f83f2a00640 1 -- 172.21.15.46:0/402371012 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e483490b40 con 0x55e48353a900 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f83f2a00640 10 client.25783 handle_command_reply: tid=0 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 client.25783 shutdown 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e483524b80 msgr2=0x55e48352ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e483524b80 0x55e48352ee90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55e483525fe0 tx=0x7f840000bfd0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 shutdown_connections 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e48353a900 0x55e48353ccf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e48352a6e0 0x55e483536410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e48352f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 --2- 172.21.15.46:0/402371012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e483524b80 0x55e48352ee90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f83f2a00640 0 client.25783 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 >> 172.21.15.46:0/402371012 conn(0x55e4835178e0 msgr2=0x55e483516ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 shutdown_connections 2024-04-17T07:59:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 1 -- 172.21.15.46:0/402371012 wait complete. 2024-04-17T07:59:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.889+0000 7f8416a2e740 20 client.25783 trim_cache size 0 max 16384 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 Processor -- start 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 -- start start 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e48352e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e483536330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec0124a0 con 0x55e483524b80 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec00fd60 con 0x55e483525570 2024-04-17T07:59:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.919+0000 7f8416a2e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec00dbb0 con 0x55e48352a6e0 2024-04-17T07:59:56.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e48352e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e48352e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56250/0 (socket says 172.21.15.46:56250) 2024-04-17T07:59:56.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 -- 172.21.15.46:0/2029826360 learned_addr learned my addr 172.21.15.46:0/2029826360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f840be00640 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e483536330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411c00640 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 -- 172.21.15.46:0/2029826360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 msgr2=0x55e48352e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 -- 172.21.15.46:0/2029826360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 msgr2=0x55e483536330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e483536330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411200640 1 -- 172.21.15.46:0/2029826360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e483518650 con 0x55e483524b80 2024-04-17T07:59:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.920+0000 7f8411c00640 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.921+0000 7f8411200640 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e48352e180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8400004720 tx=0x7f840000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11b4990288bbd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.921+0000 7f83fd600640 1 -- 172.21.15.46:0/2029826360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400004030 con 0x55e483524b80 2024-04-17T07:59:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.921+0000 7f83fd600640 1 -- 172.21.15.46:0/2029826360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840003b070 con 0x55e483524b80 2024-04-17T07:59:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.921+0000 7f83fd600640 1 -- 172.21.15.46:0/2029826360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400011b30 con 0x55e483524b80 2024-04-17T07:59:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 -- 172.21.15.46:0/2029826360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 msgr2=0x55e48352e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e48352e180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8400004720 tx=0x7f840000b390 comp rx=0 tx=0).stop 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 -- 172.21.15.46:0/2029826360 shutdown_connections 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e48352e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483525570 0x55e483536330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 --2- 172.21.15.46:0/2029826360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483524b80 0x55e48352e180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 -- 172.21.15.46:0/2029826360 >> 172.21.15.46:0/2029826360 conn(0x55e4835178e0 msgr2=0x55e48351ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.922+0000 7f8416a2e740 1 -- 172.21.15.46:0/2029826360 shutdown_connections 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.923+0000 7f8416a2e740 1 -- 172.21.15.46:0/2029826360 wait complete. 2024-04-17T07:59:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.923+0000 7f8416a2e740 1 -- start start 2024-04-17T07:59:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.923+0000 7f8416a2e740 1 Processor -- start 2024-04-17T07:59:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483524b80 0x55e4835335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483525570 0x55e483533b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8416a2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e483534b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8411200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483524b80 0x55e4835335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8416a2e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec00e220 con 0x55e483525570 2024-04-17T07:59:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8411200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483524b80 0x55e4835335f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:53630/0 (socket says 172.21.15.46:53630) 2024-04-17T07:59:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8416a2e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec00f690 con 0x55e483524b80 2024-04-17T07:59:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8411200640 1 -- 172.21.15.46:0/3132178463 learned_addr learned my addr 172.21.15.46:0/3132178463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec00e8f0 con 0x55e48352a6e0 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.924+0000 7f8416a2e740 10 client.0 _ll_register_callbacks cb 0x7f83f8009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f840be00640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483525570 0x55e483533b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411c00640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e483534b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411200640 1 -- 172.21.15.46:0/3132178463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 msgr2=0x55e483534b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411200640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e483534b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411200640 1 -- 172.21.15.46:0/3132178463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483525570 msgr2=0x55e483533b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411200640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483525570 0x55e483533b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411200640 1 -- 172.21.15.46:0/3132178463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e483535130 con 0x55e483524b80 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411c00640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e483534b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f8411200640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483524b80 0x55e4835335f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8400001eb0 tx=0x7f840000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1286e7ac651e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:59:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.925+0000 7f83f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T07:59:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f83f2a00640 1 -- 172.21.15.46:0/3132178463 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400046060 con 0x55e483524b80 2024-04-17T07:59:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f83f2a00640 1 -- 172.21.15.46:0/3132178463 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8400011c40 con 0x55e483524b80 2024-04-17T07:59:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840003b070 con 0x55e483524b80 2024-04-17T07:59:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f83f2a00640 1 -- 172.21.15.46:0/3132178463 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f840004d050 con 0x55e483524b80 2024-04-17T07:59:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f8416a2e740 10 client.25805 fetch_fsmap learned FSMap version 24 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e483536440 con 0x55e483524b80 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f83f2a00640 1 -- 172.21.15.46:0/3132178463 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840003b070 con 0x55e483524b80 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.926+0000 7f83f2a00640 1 -- 172.21.15.46:0/3132178463 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f8400041620 con 0x55e483524b80 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.927+0000 7f8416a2e740 10 client.25805 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.927+0000 7f8416a2e740 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.927+0000 7f8416a2e740 20 client.25805 populate_metadata read hostname 'smithi046' 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.927+0000 7f8416a2e740 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e4835a1290 0x55e4835a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.927+0000 7f8416a2e740 4 client.25805 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.927+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e48348a9b0 con 0x55e4835a1290 2024-04-17T07:59:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.927+0000 7f840be00640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e4835a1290 0x55e4835a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.928+0000 7f840be00640 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e4835a1290 0x55e4835a36b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428acd84434b3f02 server_cookie=47103b6e591bc749 in_seq=0 out_seq=0 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.928+0000 7f83f2a00640 10 client.25805 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.928+0000 7f83f2a00640 1 -- 172.21.15.46:0/3132178463 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e48348a9b0 con 0x55e4835a1290 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.928+0000 7f83f2a00640 10 client.25805 handle_command_reply: tid=0 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.928+0000 7f8416a2e740 1 client.25805 shutdown 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483524b80 msgr2=0x55e4835335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483524b80 0x55e4835335f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8400001eb0 tx=0x7f840000b8a0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 shutdown_connections 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55e4835a1290 0x55e4835a36b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e48352a6e0 0x55e483534b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e483525570 0x55e483533b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 --2- 172.21.15.46:0/3132178463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e483524b80 0x55e4835335f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.929+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 >> 172.21.15.46:0/3132178463 conn(0x55e4835178e0 msgr2=0x55e483516250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.930+0000 7f83f2a00640 0 client.25805 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T07:59:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.930+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 shutdown_connections 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.930+0000 7f8416a2e740 1 -- 172.21.15.46:0/3132178463 wait complete. 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.930+0000 7f8416a2e740 20 client.25805 trim_cache size 0 max 16384 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stdout: "a41451d0-4136-403e-b5b9-d06a5c76c91e": { 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-17T07:59:56.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a41451d0-4136-403e-b5b9-d06a5c76c91e", 2024-04-17T07:59:56.931 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-04-17T07:59:56.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:59:56.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T07:59:56.931 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T07:59:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83f40a3c30 msgr2=0x7f83f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83f40a3c30 0x7f83f40a60f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f83f80096c0 tx=0x7f83f8005a40 comp rx=0 tx=0).stop 2024-04-17T07:59:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 msgr2=0x7f840c1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 0x7f840c1a17f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8404036040 tx=0x7f8404004270 comp rx=0 tx=0).stop 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 shutdown_connections 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83f40a3c30 0x7f83f40a60f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c109ca0 0x7f840c1a17f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840c075f20 0x7f840c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 --2- 172.21.15.46:0/801093709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c074940 0x7f840c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.931+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 >> 172.21.15.46:0/801093709 conn(0x7f840c0fd360 msgr2=0x7f840c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.933+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 shutdown_connections 2024-04-17T07:59:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T07:59:56.933+0000 7f8412600640 1 -- 172.21.15.46:0/801093709 wait complete. 2024-04-17T07:59:56.955 INFO:tasks.cephfs.filesystem:scrub status for tag:a41451d0-4136-403e-b5b9-d06a5c76c91e - {'path': '/', 'tag': 'a41451d0-4136-403e-b5b9-d06a5c76c91e', 'options': 'recursive,force'} 2024-04-17T07:59:57.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:57 smithi062 ceph-mon[36507]: pgmap v460: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:57.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:57 smithi062 ceph-mon[36507]: scrub queued for path: / 2024-04-17T07:59:57.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:57 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:57 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T07:59:57.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:57 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:57.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:57 smithi046 ceph-mon[32038]: pgmap v460: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:57.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:57 smithi046 ceph-mon[32038]: scrub queued for path: / 2024-04-17T07:59:57.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:57 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:57.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:57 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T07:59:57.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:57 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:57.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:57 smithi053 ceph-mon[36437]: pgmap v460: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T07:59:57.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:57 smithi053 ceph-mon[36437]: scrub queued for path: / 2024-04-17T07:59:57.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:57 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:57.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:57 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T07:59:57.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:57 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T07:59:59.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 07:59:59 smithi062 ceph-mon[36507]: pgmap v461: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-17T07:59:59.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 07:59:59 smithi046 ceph-mon[32038]: pgmap v461: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-17T07:59:59.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 07:59:59 smithi053 ceph-mon[36437]: pgmap v461: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-17T08:00:00.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:00 smithi062 ceph-mon[36507]: overall HEALTH_OK 2024-04-17T08:00:00.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:00 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:00:00.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:00 smithi046 ceph-mon[32038]: overall HEALTH_OK 2024-04-17T08:00:00.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:00 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:00:00.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:00 smithi053 ceph-mon[36437]: overall HEALTH_OK 2024-04-17T08:00:00.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:00 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:00:01.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:01 smithi062 ceph-mon[36507]: pgmap v462: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T08:00:01.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:01 smithi046 ceph-mon[32038]: pgmap v462: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T08:00:01.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:01 smithi053 ceph-mon[36437]: pgmap v462: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T08:00:03.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:03 smithi062 ceph-mon[36507]: pgmap v463: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T08:00:03.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:03 smithi046 ceph-mon[32038]: pgmap v463: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T08:00:03.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:03 smithi053 ceph-mon[36437]: pgmap v463: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T08:00:05.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:05 smithi062 ceph-mon[36507]: pgmap v464: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-17T08:00:05.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:05 smithi046 ceph-mon[32038]: pgmap v464: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-17T08:00:05.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:05 smithi053 ceph-mon[36437]: pgmap v464: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-17T08:00:07.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:07 smithi062 ceph-mon[36507]: pgmap v465: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-17T08:00:07.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:07 smithi046 ceph-mon[32038]: pgmap v465: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-17T08:00:07.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:07 smithi053 ceph-mon[36437]: pgmap v465: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-17T08:00:09.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:09 smithi062 ceph-mon[36507]: pgmap v466: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-17T08:00:09.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:09 smithi046 ceph-mon[32038]: pgmap v466: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-17T08:00:09.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:09 smithi053 ceph-mon[36437]: pgmap v466: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-17T08:00:11.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:11 smithi062 ceph-mon[36507]: pgmap v467: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T08:00:11.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:11 smithi046 ceph-mon[32038]: pgmap v467: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T08:00:11.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:11 smithi053 ceph-mon[36437]: pgmap v467: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-17T08:00:13.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:13 smithi062 ceph-mon[36507]: pgmap v468: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-17T08:00:13.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:13 smithi046 ceph-mon[32038]: pgmap v468: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-17T08:00:13.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:13 smithi053 ceph-mon[36437]: pgmap v468: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-17T08:00:15.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:15 smithi062 ceph-mon[36507]: pgmap v469: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T08:00:15.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:15 smithi046 ceph-mon[32038]: pgmap v469: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T08:00:15.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:15 smithi053 ceph-mon[36437]: pgmap v469: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-17T08:00:17.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:17 smithi062 ceph-mon[36507]: pgmap v470: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T08:00:17.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:17 smithi046 ceph-mon[32038]: pgmap v470: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T08:00:17.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:17 smithi053 ceph-mon[36437]: pgmap v470: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T08:00:18.842 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 58% (7187/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 95% (11699/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-17T08:00:19.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:19 smithi062 ceph-mon[36507]: pgmap v471: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T08:00:19.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:19 smithi046 ceph-mon[32038]: pgmap v471: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T08:00:19.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:19 smithi053 ceph-mon[36437]: pgmap v471: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-04-17T08:00:19.849 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:00:19.850 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-04-17T08:00:19.850 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:00:19.850 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-04-17T08:00:19.850 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:00:19.850 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T08:00:19.850 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:00:19.850 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-17T08:00:19.863 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-17T08:00:19.926 INFO:tasks.workunit.client.0.smithi046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-17T08:00:19.934 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T08:00:19.934 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-17T08:00:20.909 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-17T08:00:21.033 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-17T08:00:21.126 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T08:00:21.131 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T08:00:21.131 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-17T08:00:21.222 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T08:00:21.231 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T08:00:21.231 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-17T08:00:21.287 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-17T08:00:21.288 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-17T08:00:21.288 DEBUG:teuthology.orchestra.run.smithi046:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-17T08:00:21.376 INFO:tasks.workunit.client.0.smithi046.stderr:+ set -e 2024-04-17T08:00:21.376 INFO:tasks.workunit.client.0.smithi046.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-17T08:00:21.383 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into 'xfstests-dev'... 2024-04-17T08:00:21.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:21 smithi062 ceph-mon[36507]: pgmap v472: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T08:00:21.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:21 smithi046 ceph-mon[32038]: pgmap v472: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T08:00:21.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:21 smithi053 ceph-mon[36437]: pgmap v472: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-17T08:00:23.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:23 smithi046 ceph-mon[32038]: pgmap v473: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T08:00:23.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:23 smithi053 ceph-mon[36437]: pgmap v473: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T08:00:23.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:23 smithi062 ceph-mon[36507]: pgmap v473: 129 pgs: 129 active+clean; 659 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-17T08:00:25.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:25 smithi046 ceph-mon[32038]: pgmap v474: 129 pgs: 129 active+clean; 788 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-04-17T08:00:25.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:25 smithi053 ceph-mon[36437]: pgmap v474: 129 pgs: 129 active+clean; 788 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-04-17T08:00:25.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:25 smithi062 ceph-mon[36507]: pgmap v474: 129 pgs: 129 active+clean; 788 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-04-17T08:00:26.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:26 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:00:26.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:26 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:00:26.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:26 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:00:26.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:00:26.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:00:27.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.195+0000 7f4e61c00640 1 -- 172.21.15.46:0/1535545564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c107ab0 msgr2=0x7f4e5c109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.195+0000 7f4e61c00640 1 --2- 172.21.15.46:0/1535545564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c107ab0 0x7f4e5c109f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009900 tx=0x7f4e5402bc40 comp rx=0 tx=0).stop 2024-04-17T08:00:27.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.195+0000 7f4e61c00640 1 -- 172.21.15.46:0/1535545564 shutdown_connections 2024-04-17T08:00:27.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.195+0000 7f4e61c00640 1 --2- 172.21.15.46:0/1535545564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c107ab0 0x7f4e5c109f00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.195+0000 7f4e61c00640 1 --2- 172.21.15.46:0/1535545564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5c105180 0x7f4e5c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.195+0000 7f4e61c00640 1 --2- 172.21.15.46:0/1535545564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c102850 0x7f4e5c104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.195+0000 7f4e61c00640 1 -- 172.21.15.46:0/1535545564 >> 172.21.15.46:0/1535545564 conn(0x7f4e5c0fc240 msgr2=0x7f4e5c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.196+0000 7f4e61c00640 1 -- 172.21.15.46:0/1535545564 shutdown_connections 2024-04-17T08:00:27.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.196+0000 7f4e61c00640 1 -- 172.21.15.46:0/1535545564 wait complete. 2024-04-17T08:00:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 Processor -- start 2024-04-17T08:00:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 -- start start 2024-04-17T08:00:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c102850 0x7f4e5c1021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c105180 0x7f4e5c100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5c107ab0 0x7f4e5c100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c10daa0 con 0x7f4e5c102850 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c10d920 con 0x7f4e5c105180 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.197+0000 7f4e61c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c10dc20 con 0x7f4e5c107ab0 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c102850 0x7f4e5c1021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c102850 0x7f4e5c1021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40376/0 (socket says 172.21.15.46:40376) 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 -- 172.21.15.46:0/161750181 learned_addr learned my addr 172.21.15.46:0/161750181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5b400640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c105180 0x7f4e5c100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e61200640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5c107ab0 0x7f4e5c100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 -- 172.21.15.46:0/161750181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5c107ab0 msgr2=0x7f4e5c100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5c107ab0 0x7f4e5c100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 -- 172.21.15.46:0/161750181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c105180 msgr2=0x7f4e5c100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c105180 0x7f4e5c100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5be00640 1 -- 172.21.15.46:0/161750181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5c1015d0 con 0x7f4e5c102850 2024-04-17T08:00:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.198+0000 7f4e5b400640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c105180 0x7f4e5c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.199+0000 7f4e5be00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c102850 0x7f4e5c1021c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00ebc0 tx=0x7f4e4c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.199+0000 7f4e58c00640 1 -- 172.21.15.46:0/161750181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c010630 con 0x7f4e5c102850 2024-04-17T08:00:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.199+0000 7f4e58c00640 1 -- 172.21.15.46:0/161750181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c010cb0 con 0x7f4e5c102850 2024-04-17T08:00:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.199+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5c1a2180 con 0x7f4e5c102850 2024-04-17T08:00:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.199+0000 7f4e58c00640 1 -- 172.21.15.46:0/161750181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c018bd0 con 0x7f4e5c102850 2024-04-17T08:00:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.199+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5c1a26d0 con 0x7f4e5c102850 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.203+0000 7f4e65fc0740 1 Processor -- start 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.203+0000 7f4e65fc0740 1 -- start start 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e58c00640 1 -- 172.21.15.46:0/161750181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e4c0107d0 con 0x7f4e5c102850 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54370 0x557062a54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a59860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a5c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5570629bb120 con 0x557062a54d60 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557062961340 con 0x557062a54370 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5570629b70e0 con 0x557062a59ed0 2024-04-17T08:00:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a59860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a59860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40378/0 (socket says 172.21.15.46:40378) 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e61200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54370 0x557062a54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e5be00640 1 -- 172.21.15.46:0/2462618158 learned_addr learned my addr 172.21.15.46:0/2462618158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.204+0000 7f4e5b400640 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a5c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e61200640 1 -- 172.21.15.46:0/2462618158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 msgr2=0x557062a5c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e61200640 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a5c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e61200640 1 -- 172.21.15.46:0/2462618158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 msgr2=0x557062a59860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e61200640 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a59860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e61200640 1 -- 172.21.15.46:0/2462618158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570627977d0 con 0x557062a54370 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e5be00640 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e61200640 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54370 0x557062a54790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e5402c180 tx=0x7f4e5402c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4438dc28d2ac88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e58c00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4e400a3bb0 0x7f4e400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e51600640 1 -- 172.21.15.46:0/2462618158 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54042070 con 0x557062a54370 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e51600640 1 -- 172.21.15.46:0/2462618158 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5402cb90 con 0x557062a54370 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e58c00640 1 -- 172.21.15.46:0/161750181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4e4c0d0d70 con 0x7f4e5c102850 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.205+0000 7f4e51600640 1 -- 172.21.15.46:0/2462618158 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540046c0 con 0x557062a54370 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e5be00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4e400a3bb0 0x7f4e400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e5be00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4e400a3bb0 0x7f4e400a6070 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00eb90 tx=0x7f4e4c0d4840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 -- 172.21.15.46:0/2462618158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54370 msgr2=0x557062a54790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54370 0x557062a54790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e5402c180 tx=0x7f4e5402c890 comp rx=0 tx=0).stop 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 -- 172.21.15.46:0/2462618158 shutdown_connections 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a5c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/2462618158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54370 0x557062a54790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 -- 172.21.15.46:0/2462618158 >> 172.21.15.46:0/2462618158 conn(0x557062a4f8e0 msgr2=0x557062a4fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.206+0000 7f4e65fc0740 1 -- 172.21.15.46:0/2462618158 shutdown_connections 2024-04-17T08:00:27.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.207+0000 7f4e65fc0740 1 -- 172.21.15.46:0/2462618158 wait complete. 2024-04-17T08:00:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.207+0000 7f4e65fc0740 1 -- start start 2024-04-17T08:00:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.207+0000 7f4e65fc0740 1 Processor -- start 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a5e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a5edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a59ed0 0x557062a65e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557062a5ce20 con 0x557062a54d60 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557062974710 con 0x557062a59ed0 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557062a5cfa0 con 0x557062a54370 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e65fc0740 10 client.0 _ll_register_callbacks cb 0x55706296faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a5edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a5e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a5e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40206/0 (socket says 172.21.15.46:40206) 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 -- 172.21.15.46:0/3807854405 learned_addr learned my addr 172.21.15.46:0/3807854405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a5edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40382/0 (socket says 172.21.15.46:40382) 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a59ed0 0x557062a65e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 -- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a59ed0 msgr2=0x557062a65e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a59ed0 0x557062a65e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 -- 172.21.15.46:0/3807854405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 msgr2=0x557062a5edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a5edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.208+0000 7f4e61200640 1 -- 172.21.15.46:0/3807854405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570626cd3f0 con 0x557062a54370 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e61200640 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a5e8b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e540096c0 tx=0x7f4e54005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d94df937b72e432 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e46000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e46000640 1 -- 172.21.15.46:0/3807854405 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54042040 con 0x557062a54370 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5402cb90 con 0x557062a54370 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e46000640 1 -- 172.21.15.46:0/3807854405 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4e5402c990 con 0x557062a54370 2024-04-17T08:00:27.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e46000640 1 -- 172.21.15.46:0/3807854405 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4e54005590 con 0x557062a54370 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e65fc0740 10 client.25801 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.209+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55706268d0f0 con 0x557062a54370 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e46000640 1 -- 172.21.15.46:0/3807854405 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5402cb90 con 0x557062a54370 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e46000640 1 -- 172.21.15.46:0/3807854405 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f4e54004ba0 con 0x557062a54370 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e65fc0740 10 client.25801 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e65fc0740 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e65fc0740 20 client.25801 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062a6a320 0x557062a6c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e65fc0740 4 client.25801 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.210+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570629c8b40 con 0x557062a6a320 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.211+0000 7f4e5b400640 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062a6a320 0x557062a6c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.211+0000 7f4e5b400640 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062a6a320 0x557062a6c710 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f115fe6df19910 server_cookie=1773c7af06a69f4b in_seq=0 out_seq=0 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.211+0000 7f4e46000640 10 client.25801 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:27.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e46000640 1 -- 172.21.15.46:0/3807854405 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570629c8b40 con 0x557062a6a320 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e46000640 10 client.25801 handle_command_reply: tid=0 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 client.25801 shutdown 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 msgr2=0x557062a5e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a5e8b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e540096c0 tx=0x7f4e54005fd0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 shutdown_connections 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062a6a320 0x557062a6c710 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a59ed0 0x557062a65e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54d60 0x557062a5edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/3807854405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a5e8b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e46000640 0 client.25801 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.212+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 >> 172.21.15.46:0/3807854405 conn(0x557062a4f8e0 msgr2=0x557062a4eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.213+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 shutdown_connections 2024-04-17T08:00:27.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.213+0000 7f4e65fc0740 1 -- 172.21.15.46:0/3807854405 wait complete. 2024-04-17T08:00:27.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.213+0000 7f4e65fc0740 20 client.25801 trim_cache size 0 max 16384 2024-04-17T08:00:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 Processor -- start 2024-04-17T08:00:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 -- start start 2024-04-17T08:00:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54370 0x557062a5ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a65df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1af410 con 0x557062a54370 2024-04-17T08:00:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1accd0 con 0x557062a54d60 2024-04-17T08:00:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.232+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1aab20 con 0x557062a59ed0 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a65df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a65df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:56696/0 (socket says 172.21.15.46:56696) 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 -- 172.21.15.46:0/489028655 learned_addr learned my addr 172.21.15.46:0/489028655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e61200640 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54370 0x557062a5ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 -- 172.21.15.46:0/489028655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 msgr2=0x557062a562f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e61200640 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 -- 172.21.15.46:0/489028655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54370 msgr2=0x557062a5ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54370 0x557062a5ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e5b400640 1 -- 172.21.15.46:0/489028655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557062a4b340 con 0x557062a54d60 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.233+0000 7f4e61200640 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.234+0000 7f4e5b400640 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a65df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c101f30 tx=0x7f4e48005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f3b9417c40678d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.234+0000 7f4e50c00640 1 -- 172.21.15.46:0/489028655 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e480035f0 con 0x557062a54d60 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.234+0000 7f4e50c00640 1 -- 172.21.15.46:0/489028655 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48003c70 con 0x557062a54d60 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.234+0000 7f4e50c00640 1 -- 172.21.15.46:0/489028655 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48020830 con 0x557062a54d60 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.234+0000 7f4e65fc0740 1 -- 172.21.15.46:0/489028655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 msgr2=0x557062a65df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.234+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a65df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c101f30 tx=0x7f4e48005f90 comp rx=0 tx=0).stop 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 -- 172.21.15.46:0/489028655 shutdown_connections 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a59ed0 0x557062a562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a65df0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/489028655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a54370 0x557062a5ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 -- 172.21.15.46:0/489028655 >> 172.21.15.46:0/489028655 conn(0x557062a4f8e0 msgr2=0x557062a52b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 -- 172.21.15.46:0/489028655 shutdown_connections 2024-04-17T08:00:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 -- 172.21.15.46:0/489028655 wait complete. 2024-04-17T08:00:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 -- start start 2024-04-17T08:00:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.235+0000 7f4e65fc0740 1 Processor -- start 2024-04-17T08:00:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a63be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e65fc0740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a59ed0 0x557062a64bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1ab190 con 0x557062a59ed0 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1ac600 con 0x557062a54d60 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e65fc0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c1ab860 con 0x557062a54370 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e65fc0740 10 client.0 _ll_register_callbacks cb 0x7f4e4c0d4710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e61200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a59ed0 0x557062a64bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e61200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e5b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a63be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e61200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40228/0 (socket says 172.21.15.46:40228) 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.236+0000 7f4e61200640 1 -- 172.21.15.46:0/4059274136 learned_addr learned my addr 172.21.15.46:0/4059274136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e5b400640 1 -- 172.21.15.46:0/4059274136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 msgr2=0x557062a636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e5b400640 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e5b400640 1 -- 172.21.15.46:0/4059274136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a59ed0 msgr2=0x557062a64bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e5b400640 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a59ed0 0x557062a64bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e5b400640 1 -- 172.21.15.46:0/4059274136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557062a651e0 con 0x557062a54d60 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e61200640 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e5b400640 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a63be0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e48014010 tx=0x7f4e48012360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61551980aed25e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e46000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e46000640 1 -- 172.21.15.46:0/4059274136 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e480035f0 con 0x557062a54d60 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e48003c70 con 0x557062a54d60 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.237+0000 7f4e46000640 1 -- 172.21.15.46:0/4059274136 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4e48003790 con 0x557062a54d60 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e46000640 1 -- 172.21.15.46:0/4059274136 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4e48003990 con 0x557062a54d60 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 10 client.25823 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557062ad0880 con 0x557062a54d60 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e46000640 1 -- 172.21.15.46:0/4059274136 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48003c70 con 0x557062a54d60 2024-04-17T08:00:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e46000640 1 -- 172.21.15.46:0/4059274136 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f4e48004a30 con 0x557062a54d60 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 10 client.25823 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 20 client.25823 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062ad1170 0x557062ad3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 4 client.25823 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.238+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557062a59cb0 con 0x557062ad1170 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.239+0000 7f4e61200640 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062ad1170 0x557062ad3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.239+0000 7f4e61200640 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062ad1170 0x557062ad3590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b3ab76eaaf2804 server_cookie=f9ff9a560fe2c05 in_seq=0 out_seq=0 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.239+0000 7f4e46000640 10 client.25823 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e46000640 1 -- 172.21.15.46:0/4059274136 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557062a59cb0 con 0x557062ad1170 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e46000640 10 client.25823 handle_command_reply: tid=0 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 client.25823 shutdown 2024-04-17T08:00:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 msgr2=0x557062a63be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a63be0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e48014010 tx=0x7f4e48012360 comp rx=0 tx=0).stop 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 shutdown_connections 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x557062ad1170 0x557062ad3590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557062a59ed0 0x557062a64bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557062a54d60 0x557062a63be0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 --2- 172.21.15.46:0/4059274136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557062a54370 0x557062a636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e46000640 0 client.25823 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 >> 172.21.15.46:0/4059274136 conn(0x557062a4f8e0 msgr2=0x557062a5ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 shutdown_connections 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 1 -- 172.21.15.46:0/4059274136 wait complete. 2024-04-17T08:00:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.240+0000 7f4e65fc0740 20 client.25823 trim_cache size 0 max 16384 2024-04-17T08:00:27.245 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:00:27.245 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-17T08:00:27.245 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-17T08:00:27.245 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4e400a3bb0 msgr2=0x7f4e400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4e400a3bb0 0x7f4e400a6070 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00eb90 tx=0x7f4e4c0d4840 comp rx=0 tx=0).stop 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c102850 msgr2=0x7f4e5c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c102850 0x7f4e5c1021c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00ebc0 tx=0x7f4e4c00c6a0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 shutdown_connections 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4e400a3bb0 0x7f4e400a6070 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e5c107ab0 0x7f4e5c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c105180 0x7f4e5c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 --2- 172.21.15.46:0/161750181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c102850 0x7f4e5c1021c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 >> 172.21.15.46:0/161750181 conn(0x7f4e5c0fc240 msgr2=0x7f4e5c103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 shutdown_connections 2024-04-17T08:00:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.241+0000 7f4e61c00640 1 -- 172.21.15.46:0/161750181 wait complete. 2024-04-17T08:00:27.259 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T08:00:27.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.477+0000 7f907f600640 1 -- 172.21.15.46:0/3667048409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105430 msgr2=0x7f90781058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.477+0000 7f907f600640 1 --2- 172.21.15.46:0/3667048409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105430 0x7f90781058b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9064009860 tx=0x7f906402bb70 comp rx=0 tx=0).stop 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.477+0000 7f907f600640 1 -- 172.21.15.46:0/3667048409 shutdown_connections 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.477+0000 7f907f600640 1 --2- 172.21.15.46:0/3667048409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078105df0 0x7f907810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.477+0000 7f907f600640 1 --2- 172.21.15.46:0/3667048409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105430 0x7f90781058b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.477+0000 7f907f600640 1 --2- 172.21.15.46:0/3667048409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104240 0x7f9078104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.477+0000 7f907f600640 1 -- 172.21.15.46:0/3667048409 >> 172.21.15.46:0/3667048409 conn(0x7f90780ff7b0 msgr2=0x7f9078101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.478+0000 7f907f600640 1 -- 172.21.15.46:0/3667048409 shutdown_connections 2024-04-17T08:00:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.478+0000 7f907f600640 1 -- 172.21.15.46:0/3667048409 wait complete. 2024-04-17T08:00:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.479+0000 7f907f600640 1 Processor -- start 2024-04-17T08:00:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.479+0000 7f907f600640 1 -- start start 2024-04-17T08:00:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.479+0000 7f907f600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078104240 0x7f907819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078105430 0x7f907819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907f600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105df0 0x7f90781a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f907810fe60 con 0x7f9078105430 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907f600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f907810fce0 con 0x7f9078105df0 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907f600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f907810ffe0 con 0x7f9078104240 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078104240 0x7f907819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078104240 0x7f907819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:40256/0 (socket says 172.21.15.46:40256) 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 -- 172.21.15.46:0/639932376 learned_addr learned my addr 172.21.15.46:0/639932376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f9077e00640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078105430 0x7f907819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907d600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105df0 0x7f90781a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 -- 172.21.15.46:0/639932376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105df0 msgr2=0x7f90781a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105df0 0x7f90781a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 -- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078105430 msgr2=0x7f907819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078105430 0x7f907819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f907cc00640 1 -- 172.21.15.46:0/639932376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90781a20e0 con 0x7f9078104240 2024-04-17T08:00:27.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.480+0000 7f9077e00640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078105430 0x7f907819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:27.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.481+0000 7f907cc00640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078104240 0x7f907819a4c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9a0 tx=0x7f906c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.481+0000 7f9075600640 1 -- 172.21.15.46:0/639932376 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00cd80 con 0x7f9078104240 2024-04-17T08:00:27.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.481+0000 7f9075600640 1 -- 172.21.15.46:0/639932376 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c010910 con 0x7f9078104240 2024-04-17T08:00:27.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.481+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90781a23d0 con 0x7f9078104240 2024-04-17T08:00:27.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.481+0000 7f9075600640 1 -- 172.21.15.46:0/639932376 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c018a30 con 0x7f9078104240 2024-04-17T08:00:27.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.481+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90781a28f0 con 0x7f9078104240 2024-04-17T08:00:27.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.482+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907810b560 con 0x7f9078104240 2024-04-17T08:00:27.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.484+0000 7f9075600640 1 -- 172.21.15.46:0/639932376 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f906c010ac0 con 0x7f9078104240 2024-04-17T08:00:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.486+0000 7f9075600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90600a3bb0 0x7f90600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.486+0000 7f9075600640 1 -- 172.21.15.46:0/639932376 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f906c0d0c70 con 0x7f9078104240 2024-04-17T08:00:27.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.486+0000 7f9077e00640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90600a3bb0 0x7f90600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:27.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.487+0000 7f9077e00640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90600a3bb0 0x7f90600a6070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f906402c180 tx=0x7f9064034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:27.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.490+0000 7f9075600640 1 -- 172.21.15.46:0/639932376 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f906c053ff0 con 0x7f9078104240 2024-04-17T08:00:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:27 smithi046 ceph-mon[32038]: pgmap v475: 129 pgs: 129 active+clean; 2.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 119 KiB/s wr, 6 op/s 2024-04-17T08:00:27.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:00:27.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:00:27.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:00:27.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:27 smithi053 ceph-mon[36437]: pgmap v475: 129 pgs: 129 active+clean; 2.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 119 KiB/s wr, 6 op/s 2024-04-17T08:00:27.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:27 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:00:27.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:27 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:00:27.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:27 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:00:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.738+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f907802d5d0 con 0x7f9078104240 2024-04-17T08:00:27.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.740+0000 7f9075600640 1 -- 172.21.15.46:0/639932376 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7f906c09d260 con 0x7f9078104240 2024-04-17T08:00:27.740 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:00:27.740 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T07:58:39.019766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T08:00:27.740 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-04-17T08:00:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.742+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90600a3bb0 msgr2=0x7f90600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.742+0000 7f907f600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90600a3bb0 0x7f90600a6070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f906402c180 tx=0x7f9064034ab0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.742+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078104240 msgr2=0x7f907819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.742+0000 7f907f600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078104240 0x7f907819a4c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9a0 tx=0x7f906c00ee60 comp rx=0 tx=0).stop 2024-04-17T08:00:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 shutdown_connections 2024-04-17T08:00:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90600a3bb0 0x7f90600a6070 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078105df0 0x7f90781a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078105430 0x7f907819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 --2- 172.21.15.46:0/639932376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9078104240 0x7f907819a4c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 >> 172.21.15.46:0/639932376 conn(0x7f90780ff7b0 msgr2=0x7f9078108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 shutdown_connections 2024-04-17T08:00:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.743+0000 7f907f600640 1 -- 172.21.15.46:0/639932376 wait complete. 2024-04-17T08:00:27.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T08:00:27.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:27 smithi062 ceph-mon[36507]: pgmap v475: 129 pgs: 129 active+clean; 2.0 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 119 KiB/s wr, 6 op/s 2024-04-17T08:00:27.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:27 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:00:27.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:27 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:00:27.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:27 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.996+0000 7f26bbe00640 1 -- 172.21.15.46:0/1585901445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 msgr2=0x7f26bc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.996+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1585901445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc076d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009be0 tx=0x7f26ac02bef0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.997+0000 7f26bbe00640 1 -- 172.21.15.46:0/1585901445 shutdown_connections 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.997+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1585901445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc078500 0x7f26bc078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.997+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1585901445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc077b60 0x7f26bc077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.997+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1585901445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc076d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.997+0000 7f26bbe00640 1 -- 172.21.15.46:0/1585901445 >> 172.21.15.46:0/1585901445 conn(0x7f26bc0ff570 msgr2=0x7f26bc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.997+0000 7f26bbe00640 1 -- 172.21.15.46:0/1585901445 shutdown_connections 2024-04-17T08:00:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.997+0000 7f26bbe00640 1 -- 172.21.15.46:0/1585901445 wait complete. 2024-04-17T08:00:27.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.998+0000 7f26bbe00640 1 Processor -- start 2024-04-17T08:00:27.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.998+0000 7f26bbe00640 1 -- start start 2024-04-17T08:00:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.998+0000 7f26bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26bbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc077b60 0x7f26bc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc078500 0x7f26bc1a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26bbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1140b0 con 0x7f26bc078500 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26bbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc113f30 con 0x7f26bc077b60 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26bbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc114230 con 0x7f26bc076920 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc078500 0x7f26bc1a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55346/0 (socket says 172.21.15.46:55346) 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 -- 172.21.15.46:0/1589823499 learned_addr learned my addr 172.21.15.46:0/1589823499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26b2000640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc077b60 0x7f26bc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 -- 172.21.15.46:0/1589823499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc077b60 msgr2=0x7f26bc19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc077b60 0x7f26bc19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 -- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc078500 msgr2=0x7f26bc1a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc078500 0x7f26bc1a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:27.999+0000 7f26baa00640 1 -- 172.21.15.46:0/1589823499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc1a6300 con 0x7f26bc076920 2024-04-17T08:00:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.000+0000 7f26baa00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc19e6e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f26ac000f80 tx=0x7f26ac0337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.000+0000 7f26b3e00640 1 -- 172.21.15.46:0/1589823499 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac043020 con 0x7f26bc076920 2024-04-17T08:00:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.000+0000 7f26b3e00640 1 -- 172.21.15.46:0/1589823499 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac02ca60 con 0x7f26bc076920 2024-04-17T08:00:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.000+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc1a6590 con 0x7f26bc076920 2024-04-17T08:00:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.000+0000 7f26b3e00640 1 -- 172.21.15.46:0/1589823499 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac034690 con 0x7f26bc076920 2024-04-17T08:00:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.000+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc1a6b30 con 0x7f26bc076920 2024-04-17T08:00:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 Processor -- start 2024-04-17T08:00:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26b3e00640 1 -- 172.21.15.46:0/1589823499 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f26ac046020 con 0x7f26bc076920 2024-04-17T08:00:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 -- start start 2024-04-17T08:00:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90160 0x55e7f8a90580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a95650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a98150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e7f89f6020 con 0x55e7f8a90160 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e7f899c460 con 0x55e7f8a95cc0 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26c4450740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e7f89f1fe0 con 0x55e7f8a90b50 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90160 0x55e7f8a90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90160 0x55e7f8a90580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36518/0 (socket says 172.21.15.46:36518) 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.004+0000 7f26bb400640 1 -- 172.21.15.46:0/566966099 learned_addr learned my addr 172.21.15.46:0/566966099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26baa00640 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a95650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26b2000640 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a98150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26bb400640 1 -- 172.21.15.46:0/566966099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 msgr2=0x55e7f8a95650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26bb400640 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a95650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26bb400640 1 -- 172.21.15.46:0/566966099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a95cc0 msgr2=0x55e7f8a98150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26bb400640 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a98150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26bb400640 1 -- 172.21.15.46:0/566966099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f89a4040 con 0x55e7f8a90160 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26b2000640 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a98150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26b3e00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26a00a3c30 0x7f26a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.005+0000 7f26bb400640 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90160 0x55e7f8a90580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009c40 tx=0x7f26b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef113a4eec0e3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26b3e00640 1 -- 172.21.15.46:0/1589823499 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f26ac0f2d70 con 0x7f26bc076920 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26baa00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26a00a3c30 0x7f26a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26abe00640 1 -- 172.21.15.46:0/566966099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4004230 con 0x55e7f8a90160 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26abe00640 1 -- 172.21.15.46:0/566966099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b40043d0 con 0x55e7f8a90160 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26abe00640 1 -- 172.21.15.46:0/566966099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4019790 con 0x55e7f8a90160 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26c4450740 1 -- 172.21.15.46:0/566966099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90160 msgr2=0x55e7f8a90580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26c4450740 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90160 0x55e7f8a90580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009c40 tx=0x7f26b4011810 comp rx=0 tx=0).stop 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.006+0000 7f26baa00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26a00a3c30 0x7f26a00a60f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0096f0 tx=0x7f26ac0f67b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 -- 172.21.15.46:0/566966099 shutdown_connections 2024-04-17T08:00:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a98150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a95650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 --2- 172.21.15.46:0/566966099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90160 0x55e7f8a90580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 -- 172.21.15.46:0/566966099 >> 172.21.15.46:0/566966099 conn(0x55e7f8a8b6d0 msgr2=0x55e7f8a8bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 -- 172.21.15.46:0/566966099 shutdown_connections 2024-04-17T08:00:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 -- 172.21.15.46:0/566966099 wait complete. 2024-04-17T08:00:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 -- start start 2024-04-17T08:00:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.007+0000 7f26c4450740 1 Processor -- start 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26c4450740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26c4450740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26c4450740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26c4450740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e7f8a98c70 con 0x55e7f8a90b50 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26c4450740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e7f89af830 con 0x55e7f8a90160 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26c4450740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e7f8a98df0 con 0x55e7f8a95cc0 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26c4450740 10 client.0 _ll_register_callbacks cb 0x55e7f897dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36526/0 (socket says 172.21.15.46:36526) 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26b2000640 1 -- 172.21.15.46:0/237678779 learned_addr learned my addr 172.21.15.46:0/237678779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.008+0000 7f26bb400640 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26b2000640 1 -- 172.21.15.46:0/237678779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a95cc0 msgr2=0x55e7f8aa1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26b2000640 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26b2000640 1 -- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 msgr2=0x55e7f8a9a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26b2000640 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26b2000640 1 -- 172.21.15.46:0/237678779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f89b3920 con 0x55e7f8a90b50 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26bb400640 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26b2000640 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9ac40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f26a4012700 tx=0x7f26a4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f02cef886df219 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26a8c00640 1 -- 172.21.15.46:0/237678779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4004030 con 0x55e7f8a90b50 2024-04-17T08:00:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26a8c00640 1 -- 172.21.15.46:0/237678779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f26a40041d0 con 0x55e7f8a90b50 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.009+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b40043d0 con 0x55e7f8a90b50 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26a8c00640 1 -- 172.21.15.46:0/237678779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26a4004410 con 0x55e7f8a90b50 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 10 client.15693 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7f89817f0 con 0x55e7f8a90b50 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26a8c00640 1 -- 172.21.15.46:0/237678779 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b40043d0 con 0x55e7f8a90b50 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26a8c00640 1 -- 172.21.15.46:0/237678779 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f26a4003bf0 con 0x55e7f8a90b50 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 10 client.15693 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 10 client.15693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 20 client.15693 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8aa62e0 0x55e7f8aa8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 4 client.15693 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.010+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7f8a03a40 con 0x55e7f8aa62e0 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.011+0000 7f26bb400640 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8aa62e0 0x55e7f8aa8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.011+0000 7f26bb400640 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8aa62e0 0x55e7f8aa8740 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b19a6107d649e7ae server_cookie=c45d1d0ae8ca1e71 in_seq=0 out_seq=0 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.011+0000 7f26a8c00640 10 client.15693 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26a8c00640 1 -- 172.21.15.46:0/237678779 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7f8a03a40 con 0x55e7f8aa62e0 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26a8c00640 10 client.15693 handle_command_reply: tid=0 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 client.15693 shutdown 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90b50 msgr2=0x55e7f8a9ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9ac40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f26a4012700 tx=0x7f26a4012bc0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 shutdown_connections 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8aa62e0 0x55e7f8aa8740 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9ac40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26a8c00640 0 client.15693 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 --2- 172.21.15.46:0/237678779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.012+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 >> 172.21.15.46:0/237678779 conn(0x55e7f8a8b6d0 msgr2=0x55e7f8a8a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.013+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 shutdown_connections 2024-04-17T08:00:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.013+0000 7f26c4450740 1 -- 172.21.15.46:0/237678779 wait complete. 2024-04-17T08:00:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.013+0000 7f26c4450740 20 client.15693 trim_cache size 0 max 16384 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 Processor -- start 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 -- start start 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a8bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8aa1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1b3870 con 0x55e7f8a95cc0 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1b1130 con 0x55e7f8a90160 2024-04-17T08:00:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26c4450740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1aef80 con 0x55e7f8a90b50 2024-04-17T08:00:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26b2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8aa1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a8bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26b2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8aa1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55374/0 (socket says 172.21.15.46:55374) 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26b2000640 1 -- 172.21.15.46:0/1207632949 learned_addr learned my addr 172.21.15.46:0/1207632949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.028+0000 7f26bb400640 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.029+0000 7f26b2000640 1 -- 172.21.15.46:0/1207632949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 msgr2=0x55e7f8a8bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.029+0000 7f26b2000640 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a8bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.029+0000 7f26b2000640 1 -- 172.21.15.46:0/1207632949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 msgr2=0x55e7f8a920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.029+0000 7f26b2000640 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.029+0000 7f26b2000640 1 -- 172.21.15.46:0/1207632949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f87083f0 con 0x55e7f8a90b50 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.029+0000 7f26b2000640 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8aa1c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26a4010670 tx=0x7f26a4005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a8aae908386242 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.029+0000 7f26ab400640 1 -- 172.21.15.46:0/1207632949 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a402e070 con 0x55e7f8a90b50 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26ab400640 1 -- 172.21.15.46:0/1207632949 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a400fc70 con 0x55e7f8a90b50 2024-04-17T08:00:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26ab400640 1 -- 172.21.15.46:0/1207632949 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4029440 con 0x55e7f8a90b50 2024-04-17T08:00:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 -- 172.21.15.46:0/1207632949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 msgr2=0x55e7f8aa1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8aa1c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26a4010670 tx=0x7f26a4005640 comp rx=0 tx=0).stop 2024-04-17T08:00:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 -- 172.21.15.46:0/1207632949 shutdown_connections 2024-04-17T08:00:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8a920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8aa1c40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 --2- 172.21.15.46:0/1207632949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a8bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 -- 172.21.15.46:0/1207632949 >> 172.21.15.46:0/1207632949 conn(0x55e7f8a8b6d0 msgr2=0x55e7f8a991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 -- 172.21.15.46:0/1207632949 shutdown_connections 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.030+0000 7f26c4450740 1 -- 172.21.15.46:0/1207632949 wait complete. 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.031+0000 7f26c4450740 1 -- start start 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.031+0000 7f26c4450740 1 Processor -- start 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.031+0000 7f26c4450740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.031+0000 7f26c4450740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26c4450740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26c4450740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1af5f0 con 0x55e7f8a95cc0 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26c4450740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1b0a60 con 0x55e7f8a90160 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26c4450740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1afcc0 con 0x55e7f8a90b50 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26c4450740 10 client.0 _ll_register_callbacks cb 0x7f26a4009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55384/0 (socket says 172.21.15.46:55384) 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 -- 172.21.15.46:0/452742359 learned_addr learned my addr 172.21.15.46:0/452742359 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36536/0 (socket says 172.21.15.46:36536) 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26bb400640 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 -- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 msgr2=0x55e7f8a9edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 -- 172.21.15.46:0/452742359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 msgr2=0x55e7f8aa0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26b2000640 1 -- 172.21.15.46:0/452742359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f87d27d0 con 0x55e7f8a90b50 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.032+0000 7f26bb400640 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26b2000640 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9f310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f26a4012420 tx=0x7f26a4005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=626237aa32e1c24d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26a8c00640 1 -- 172.21.15.46:0/452742359 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4037060 con 0x55e7f8a90b50 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26a8c00640 1 -- 172.21.15.46:0/452742359 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f26a400f790 con 0x55e7f8a90b50 2024-04-17T08:00:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a400fc70 con 0x55e7f8a90b50 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26a8c00640 1 -- 172.21.15.46:0/452742359 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26a402e070 con 0x55e7f8a90b50 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26c4450740 10 client.25828 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.033+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7f8aa1a40 con 0x55e7f8a90b50 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26a8c00640 1 -- 172.21.15.46:0/452742359 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a400fc70 con 0x55e7f8a90b50 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26a8c00640 1 -- 172.21.15.46:0/452742359 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f26a4032410 con 0x55e7f8a90b50 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26c4450740 10 client.25828 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26c4450740 10 client.25828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26c4450740 20 client.25828 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26c4450740 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8b0c960 0x55e7f8b0ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26c4450740 4 client.25828 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7f89fd8b0 con 0x55e7f8b0c960 2024-04-17T08:00:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.034+0000 7f26bb400640 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8b0c960 0x55e7f8b0ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26bb400640 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8b0c960 0x55e7f8b0ed80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a55a0b0e5fb212d8 server_cookie=6ce82836c7b13b35 in_seq=0 out_seq=0 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26a8c00640 10 client.25828 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26a8c00640 1 -- 172.21.15.46:0/452742359 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7f89fd8b0 con 0x55e7f8b0c960 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26a8c00640 10 client.25828 handle_command_reply: tid=0 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 client.25828 shutdown 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 msgr2=0x55e7f8a9f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9f310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f26a4012420 tx=0x7f26a4005b90 comp rx=0 tx=0).stop 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 shutdown_connections 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55e7f8b0c960 0x55e7f8b0ed80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7f8a95cc0 0x55e7f8aa0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e7f8a90b50 0x55e7f8a9f310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26a8c00640 0 client.25828 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 --2- 172.21.15.46:0/452742359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7f8a90160 0x55e7f8a9edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 >> 172.21.15.46:0/452742359 conn(0x55e7f8a8b6d0 msgr2=0x55e7f8a8bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.035+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 shutdown_connections 2024-04-17T08:00:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26c4450740 1 -- 172.21.15.46:0/452742359 wait complete. 2024-04-17T08:00:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26c4450740 20 client.25828 trim_cache size 0 max 16384 2024-04-17T08:00:28.040 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26a00a3c30 msgr2=0x7f26a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26a00a3c30 0x7f26a00a60f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0096f0 tx=0x7f26ac0f67b0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 msgr2=0x7f26bc19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc19e6e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f26ac000f80 tx=0x7f26ac0337b0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 shutdown_connections 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26a00a3c30 0x7f26a00a60f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26bc078500 0x7f26bc1a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26bc077b60 0x7f26bc19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 --2- 172.21.15.46:0/1589823499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26bc076920 0x7f26bc19e6e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.036+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 >> 172.21.15.46:0/1589823499 conn(0x7f26bc0ff570 msgr2=0x7f26bc101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.037+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 shutdown_connections 2024-04-17T08:00:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.037+0000 7f26bbe00640 1 -- 172.21.15.46:0/1589823499 wait complete. 2024-04-17T08:00:28.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T08:00:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.274+0000 7f7c83200640 1 -- 172.21.15.46:0/3872470889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c84102fe0 msgr2=0x7f7c84103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.274+0000 7f7c83200640 1 --2- 172.21.15.46:0/3872470889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c84102fe0 0x7f7c84103460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009900 tx=0x7f7c7002bc50 comp rx=0 tx=0).stop 2024-04-17T08:00:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.275+0000 7f7c83200640 1 -- 172.21.15.46:0/3872470889 shutdown_connections 2024-04-17T08:00:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.275+0000 7f7c83200640 1 --2- 172.21.15.46:0/3872470889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c841039a0 0x7f7c8410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.275+0000 7f7c83200640 1 --2- 172.21.15.46:0/3872470889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c84102fe0 0x7f7c84103460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.275+0000 7f7c83200640 1 --2- 172.21.15.46:0/3872470889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c84101df0 0x7f7c84102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.275+0000 7f7c83200640 1 -- 172.21.15.46:0/3872470889 >> 172.21.15.46:0/3872470889 conn(0x7f7c840fd360 msgr2=0x7f7c840ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.275+0000 7f7c83200640 1 -- 172.21.15.46:0/3872470889 shutdown_connections 2024-04-17T08:00:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.275+0000 7f7c83200640 1 -- 172.21.15.46:0/3872470889 wait complete. 2024-04-17T08:00:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.276+0000 7f7c83200640 1 Processor -- start 2024-04-17T08:00:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.276+0000 7f7c83200640 1 -- start start 2024-04-17T08:00:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c83200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c84101df0 0x7f7c8419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c83200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c84102fe0 0x7f7c8419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c83200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c841039a0 0x7f7c841a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c83200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410daa0 con 0x7f7c841039a0 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c83200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410d920 con 0x7f7c84101df0 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c83200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410dc20 con 0x7f7c84102fe0 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c84101df0 0x7f7c8419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c82800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c841039a0 0x7f7c841a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c84101df0 0x7f7c8419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51580/0 (socket says 172.21.15.46:51580) 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 -- 172.21.15.46:0/3760910597 learned_addr learned my addr 172.21.15.46:0/3760910597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81400640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c84102fe0 0x7f7c8419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 -- 172.21.15.46:0/3760910597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c84102fe0 msgr2=0x7f7c8419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c84102fe0 0x7f7c8419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 -- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c841039a0 msgr2=0x7f7c841a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c841039a0 0x7f7c841a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.277+0000 7f7c81e00640 1 -- 172.21.15.46:0/3760910597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c841a1f80 con 0x7f7c84101df0 2024-04-17T08:00:28.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.278+0000 7f7c81e00640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c84101df0 0x7f7c8419a360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400ebc0 tx=0x7f7c7400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.278+0000 7f7c7aa00640 1 -- 172.21.15.46:0/3760910597 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74003970 con 0x7f7c84101df0 2024-04-17T08:00:28.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.278+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c841a2270 con 0x7f7c84101df0 2024-04-17T08:00:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.278+0000 7f7c7aa00640 1 -- 172.21.15.46:0/3760910597 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c74004030 con 0x7f7c84101df0 2024-04-17T08:00:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.278+0000 7f7c7aa00640 1 -- 172.21.15.46:0/3760910597 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74019740 con 0x7f7c84101df0 2024-04-17T08:00:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.278+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c841a27e0 con 0x7f7c84101df0 2024-04-17T08:00:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.281+0000 7f7c8b7b5740 1 Processor -- start 2024-04-17T08:00:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c8b7b5740 1 -- start start 2024-04-17T08:00:28.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c7aa00640 1 -- 172.21.15.46:0/3760910597 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7c740041e0 con 0x7f7c84101df0 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5cfa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5cff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d02150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5c60020 con 0x55d8e5cffcc0 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5c06460 con 0x55d8e5cfa160 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5c5bfe0 con 0x55d8e5cfab50 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c82800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5cfa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c82800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5cfa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51586/0 (socket says 172.21.15.46:51586) 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c82800640 1 -- 172.21.15.46:0/2156989523 learned_addr learned my addr 172.21.15.46:0/2156989523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c81400640 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d02150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.282+0000 7f7c81e00640 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5cff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c82800640 1 -- 172.21.15.46:0/2156989523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 msgr2=0x55d8e5cff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c82800640 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5cff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c82800640 1 -- 172.21.15.46:0/2156989523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 msgr2=0x55d8e5d02150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c82800640 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d02150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c82800640 1 -- 172.21.15.46:0/2156989523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e5c0e040 con 0x55d8e5cfa160 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c82800640 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5cfa580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009c90 tx=0x7f7c7c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b74c93b52f476a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c6f000640 1 -- 172.21.15.46:0/2156989523 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c004230 con 0x55d8e5cfa160 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c6f000640 1 -- 172.21.15.46:0/2156989523 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c009e40 con 0x55d8e5cfa160 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c6f000640 1 -- 172.21.15.46:0/2156989523 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0197e0 con 0x55d8e5cfa160 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.283+0000 7f7c7aa00640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7c680a3c30 0x7f7c680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c7aa00640 1 -- 172.21.15.46:0/3760910597 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7c740d10d0 con 0x7f7c84101df0 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c81e00640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7c680a3c30 0x7f7c680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2156989523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 msgr2=0x55d8e5cfa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5cfa580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009c90 tx=0x7f7c7c011810 comp rx=0 tx=0).stop 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2156989523 shutdown_connections 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d02150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5cff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2156989523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5cfa580 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2156989523 >> 172.21.15.46:0/2156989523 conn(0x55d8e5cf56d0 msgr2=0x55d8e5cf5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2156989523 shutdown_connections 2024-04-17T08:00:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2156989523 wait complete. 2024-04-17T08:00:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.284+0000 7f7c81e00640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7c680a3c30 0x7f7c680a60f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7c74099420 tx=0x7f7c740d4840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.285+0000 7f7c8b7b5740 1 -- start start 2024-04-17T08:00:28.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.285+0000 7f7c8b7b5740 1 Processor -- start 2024-04-17T08:00:28.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.285+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5d04620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.285+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d04b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.285+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5d02c70 con 0x55d8e5cffcc0 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5c19830 con 0x55d8e5cfab50 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5d02df0 con 0x55d8e5cfa160 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c8b7b5740 10 client.0 _ll_register_callbacks cb 0x55d8e5be7fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c82800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5d04620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c81400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d04b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c82800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5d04620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55420/0 (socket says 172.21.15.46:55420) 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c81400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d04b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51594/0 (socket says 172.21.15.46:51594) 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c82800640 1 -- 172.21.15.46:0/1797360754 learned_addr learned my addr 172.21.15.46:0/1797360754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c82800640 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c81400640 1 -- 172.21.15.46:0/1797360754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 msgr2=0x55d8e5d04620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c81400640 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5d04620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c81400640 1 -- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 msgr2=0x55d8e5d0bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c81400640 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.286+0000 7f7c81400640 1 -- 172.21.15.46:0/1797360754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e5c1d920 con 0x55d8e5cfab50 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c82800640 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5d04620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c81400640 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d04b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c70036040 tx=0x7f7c70004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fdb164f11c0ece0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c63400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c63400640 1 -- 172.21.15.46:0/1797360754 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70005d50 con 0x55d8e5cfab50 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8e5c6da40 con 0x55d8e5cfab50 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c63400640 1 -- 172.21.15.46:0/1797360754 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7c700048c0 con 0x55d8e5cfab50 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c63400640 1 -- 172.21.15.46:0/1797360754 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c70004ae0 con 0x55d8e5cfab50 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c8b7b5740 10 client.25859 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.287+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d8e5beb7f0 con 0x55d8e5cfab50 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c63400640 1 -- 172.21.15.46:0/1797360754 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d8e5c6da40 con 0x55d8e5cfab50 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c63400640 1 -- 172.21.15.46:0/1797360754 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f7c70046070 con 0x55d8e5cfab50 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c8b7b5740 10 client.25859 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c8b7b5740 10 client.25859 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c8b7b5740 20 client.25859 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d10080 0x55d8e5d124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c8b7b5740 4 client.25859 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e5c678b0 con 0x55d8e5d10080 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.288+0000 7f7c82800640 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d10080 0x55d8e5d124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.289+0000 7f7c82800640 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d10080 0x55d8e5d124e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc4ea8cc2f203ef6 server_cookie=8cddee0548f5b87b in_seq=0 out_seq=0 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.289+0000 7f7c63400640 10 client.25859 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c63400640 1 -- 172.21.15.46:0/1797360754 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8e5c678b0 con 0x55d8e5d10080 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c63400640 10 client.25859 handle_command_reply: tid=0 2024-04-17T08:00:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 client.25859 shutdown 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 msgr2=0x55d8e5d04b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d04b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c70036040 tx=0x7f7c70004250 comp rx=0 tx=0).stop 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 shutdown_connections 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d10080 0x55d8e5d124e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d04b60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c63400640 0 client.25859 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1797360754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5d04620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 >> 172.21.15.46:0/1797360754 conn(0x55d8e5cf56d0 msgr2=0x55d8e5cf4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 shutdown_connections 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1797360754 wait complete. 2024-04-17T08:00:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.290+0000 7f7c8b7b5740 20 client.25859 trim_cache size 0 max 16384 2024-04-17T08:00:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.305+0000 7f7c8b7b5740 1 Processor -- start 2024-04-17T08:00:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.305+0000 7f7c8b7b5740 1 -- start start 2024-04-17T08:00:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5cf40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d0bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5cf45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841af560 con 0x55d8e5cffcc0 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ace20 con 0x55d8e5cfab50 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841aac70 con 0x55d8e5cfa160 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c82800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5cf40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c82800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5cf40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55424/0 (socket says 172.21.15.46:55424) 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c82800640 1 -- 172.21.15.46:0/2824411971 learned_addr learned my addr 172.21.15.46:0/2824411971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c81400640 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d0bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c82800640 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5cf45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c81400640 1 -- 172.21.15.46:0/2824411971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 msgr2=0x55d8e5cf40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c81400640 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5cf40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c81400640 1 -- 172.21.15.46:0/2824411971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 msgr2=0x55d8e5cf45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.306+0000 7f7c81400640 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5cf45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.307+0000 7f7c81400640 1 -- 172.21.15.46:0/2824411971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e59723f0 con 0x55d8e5cfab50 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.307+0000 7f7c81400640 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d0bb60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7c7002c190 tx=0x7f7c7002ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1609bb595c1ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.307+0000 7f7c6e600640 1 -- 172.21.15.46:0/2824411971 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70005d50 con 0x55d8e5cfab50 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.307+0000 7f7c6e600640 1 -- 172.21.15.46:0/2824411971 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c70005850 con 0x55d8e5cfab50 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.307+0000 7f7c6e600640 1 -- 172.21.15.46:0/2824411971 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70004920 con 0x55d8e5cfab50 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2824411971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 msgr2=0x55d8e5d0bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d0bb60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7c7002c190 tx=0x7f7c7002ccb0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2824411971 shutdown_connections 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5cf45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfab50 0x55d8e5d0bb60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/2824411971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfa160 0x55d8e5cf40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2824411971 >> 172.21.15.46:0/2824411971 conn(0x55d8e5cf56d0 msgr2=0x55d8e5cfc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2824411971 shutdown_connections 2024-04-17T08:00:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/2824411971 wait complete. 2024-04-17T08:00:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.308+0000 7f7c8b7b5740 1 -- start start 2024-04-17T08:00:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 1 Processor -- start 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5d08e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5d09380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ab2e0 con 0x55d8e5cffcc0 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ac750 con 0x55d8e5cfa160 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ab9b0 con 0x55d8e5cfab50 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c8b7b5740 10 client.0 _ll_register_callbacks cb 0x7f7c7002ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.309+0000 7f7c82800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5d08e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c82800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5d08e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51606/0 (socket says 172.21.15.46:51606) 2024-04-17T08:00:28.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c81400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5d09380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c82800640 1 -- 172.21.15.46:0/1772386787 learned_addr learned my addr 172.21.15.46:0/1772386787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c82800640 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c81400640 1 -- 172.21.15.46:0/1772386787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 msgr2=0x55d8e5d08e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c81400640 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5d08e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c81400640 1 -- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 msgr2=0x55d8e5d0a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c81400640 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c81400640 1 -- 172.21.15.46:0/1772386787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e5a3c7d0 con 0x55d8e5cfab50 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c82800640 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5d08e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c81400640 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5d09380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c7002c260 tx=0x7f7c70003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=431ab13b8e69f921 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.310+0000 7f7c63400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c63400640 1 -- 172.21.15.46:0/1772386787 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70005d50 con 0x55d8e5cfab50 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c63400640 1 -- 172.21.15.46:0/1772386787 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7c700050e0 con 0x55d8e5cfab50 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c70005850 con 0x55d8e5cfab50 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c63400640 1 -- 172.21.15.46:0/1772386787 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c70066050 con 0x55d8e5cfab50 2024-04-17T08:00:28.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 10 client.25846 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d8e5d0bab0 con 0x55d8e5cfab50 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c63400640 1 -- 172.21.15.46:0/1772386787 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c70005850 con 0x55d8e5cfab50 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c63400640 1 -- 172.21.15.46:0/1772386787 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f7c70055030 con 0x55d8e5cfab50 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 10 client.25846 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 10 client.25846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 20 client.25846 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d7f120 0x55d8e5d81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 4 client.25846 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.311+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8e5be3570 con 0x55d8e5d7f120 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.312+0000 7f7c82800640 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d7f120 0x55d8e5d81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.312+0000 7f7c82800640 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d7f120 0x55d8e5d81510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=247d75e4bf98aa47 server_cookie=d514932d1f45fd10 in_seq=0 out_seq=0 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.312+0000 7f7c63400640 10 client.25846 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c63400640 1 -- 172.21.15.46:0/1772386787 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8e5be3570 con 0x55d8e5d7f120 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c63400640 10 client.25846 handle_command_reply: tid=0 2024-04-17T08:00:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 client.25846 shutdown 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 msgr2=0x55d8e5d09380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5d09380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c7002c260 tx=0x7f7c70003db0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 shutdown_connections 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55d8e5d7f120 0x55d8e5d81510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8e5cffcc0 0x55d8e5d0a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8e5cfab50 0x55d8e5d09380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 --2- 172.21.15.46:0/1772386787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d8e5cfa160 0x55d8e5d08e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c63400640 0 client.25846 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 >> 172.21.15.46:0/1772386787 conn(0x55d8e5cf56d0 msgr2=0x55d8e5cf8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 shutdown_connections 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 1 -- 172.21.15.46:0/1772386787 wait complete. 2024-04-17T08:00:28.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.313+0000 7f7c8b7b5740 20 client.25846 trim_cache size 0 max 16384 2024-04-17T08:00:28.319 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:00:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7c680a3c30 msgr2=0x7f7c680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7c680a3c30 0x7f7c680a60f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7c74099420 tx=0x7f7c740d4840 comp rx=0 tx=0).stop 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c84101df0 msgr2=0x7f7c8419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c84101df0 0x7f7c8419a360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400ebc0 tx=0x7f7c7400c2d0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 shutdown_connections 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7c680a3c30 0x7f7c680a60f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c841039a0 0x7f7c841a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c84102fe0 0x7f7c8419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 --2- 172.21.15.46:0/3760910597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c84101df0 0x7f7c8419a360 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 >> 172.21.15.46:0/3760910597 conn(0x7f7c840fd360 msgr2=0x7f7c84074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 shutdown_connections 2024-04-17T08:00:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.314+0000 7f7c83200640 1 -- 172.21.15.46:0/3760910597 wait complete. 2024-04-17T08:00:28.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T08:00:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.552+0000 7fa3e0000640 1 -- 172.21.15.46:0/976076150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d81041c0 msgr2=0x7fa3d81045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.552+0000 7fa3e0000640 1 --2- 172.21.15.46:0/976076150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d81041c0 0x7fa3d81045e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009900 tx=0x7fa3cc02bc30 comp rx=0 tx=0).stop 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.552+0000 7fa3e0000640 1 -- 172.21.15.46:0/976076150 shutdown_connections 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.552+0000 7fa3e0000640 1 --2- 172.21.15.46:0/976076150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3d8105d70 0x7fa3d810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.552+0000 7fa3e0000640 1 --2- 172.21.15.46:0/976076150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d81053b0 0x7fa3d8105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.552+0000 7fa3e0000640 1 --2- 172.21.15.46:0/976076150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d81041c0 0x7fa3d81045e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.552+0000 7fa3e0000640 1 -- 172.21.15.46:0/976076150 >> 172.21.15.46:0/976076150 conn(0x7fa3d80ff750 msgr2=0x7fa3d8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.553+0000 7fa3e0000640 1 -- 172.21.15.46:0/976076150 shutdown_connections 2024-04-17T08:00:28.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.553+0000 7fa3e0000640 1 -- 172.21.15.46:0/976076150 wait complete. 2024-04-17T08:00:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.553+0000 7fa3e0000640 1 Processor -- start 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3e0000640 1 -- start start 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3e0000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3d81041c0 0x7fa3d819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3e0000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d81053b0 0x7fa3d819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3e0000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8105d70 0x7fa3d81a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3e0000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d810fea0 con 0x7fa3d81053b0 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3e0000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d810fd20 con 0x7fa3d8105d70 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3e0000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8110020 con 0x7fa3d81041c0 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3dcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d81053b0 0x7fa3d819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3dd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3d81041c0 0x7fa3d819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3de000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8105d70 0x7fa3d81a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3dcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d81053b0 0x7fa3d819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36604/0 (socket says 172.21.15.46:36604) 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.554+0000 7fa3dcc00640 1 -- 172.21.15.46:0/2457797108 learned_addr learned my addr 172.21.15.46:0/2457797108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.555+0000 7fa3de000640 1 -- 172.21.15.46:0/2457797108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3d81041c0 msgr2=0x7fa3d819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.555+0000 7fa3de000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3d81041c0 0x7fa3d819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.555+0000 7fa3de000640 1 -- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d81053b0 msgr2=0x7fa3d819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.555+0000 7fa3de000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d81053b0 0x7fa3d819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.555+0000 7fa3de000640 1 -- 172.21.15.46:0/2457797108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d81a2140 con 0x7fa3d8105d70 2024-04-17T08:00:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.555+0000 7fa3de000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8105d70 0x7fa3d81a1ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009fd0 tx=0x7fa3d000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.555+0000 7fa3d6000640 1 -- 172.21.15.46:0/2457797108 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000ce20 con 0x7fa3d8105d70 2024-04-17T08:00:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.556+0000 7fa3d6000640 1 -- 172.21.15.46:0/2457797108 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0005020 con 0x7fa3d8105d70 2024-04-17T08:00:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.556+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d81a2430 con 0x7fa3d8105d70 2024-04-17T08:00:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.556+0000 7fa3d6000640 1 -- 172.21.15.46:0/2457797108 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0010720 con 0x7fa3d8105d70 2024-04-17T08:00:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.556+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81a29d0 con 0x7fa3d8105d70 2024-04-17T08:00:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3d6000640 1 -- 172.21.15.46:0/2457797108 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa3d0005540 con 0x7fa3d8105d70 2024-04-17T08:00:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 Processor -- start 2024-04-17T08:00:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 -- start start 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b880fa0 0x560c6b8813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b886490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b886b00 0x560c6b888f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560c6b7e7020 con 0x560c6b886b00 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c6b78d460 con 0x560c6b880fa0 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.559+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560c6b7e2fe0 con 0x560c6b881990 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b886490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b886490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55460/0 (socket says 172.21.15.46:55460) 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 -- 172.21.15.46:0/1822145012 learned_addr learned my addr 172.21.15.46:0/1822145012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b886b00 0x560c6b888f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b880fa0 0x560c6b8813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 -- 172.21.15.46:0/1822145012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b880fa0 msgr2=0x560c6b8813c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b880fa0 0x560c6b8813c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 -- 172.21.15.46:0/1822145012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b886b00 msgr2=0x560c6b888f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b886b00 0x560c6b888f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.560+0000 7fa3dd600640 1 -- 172.21.15.46:0/1822145012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c6b795040 con 0x560c6b881990 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.561+0000 7fa3dd600640 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b886490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10f0f4de332195b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.561+0000 7fa3d6000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa3c00a3c30 0x7fa3c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.561+0000 7fa3d6000640 1 -- 172.21.15.46:0/2457797108 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa3d00d0cf0 con 0x7fa3d8105d70 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.561+0000 7fa3c6600640 1 -- 172.21.15.46:0/1822145012 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc02c980 con 0x560c6b881990 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.561+0000 7fa3c6600640 1 -- 172.21.15.46:0/1822145012 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3cc02cb20 con 0x560c6b881990 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.561+0000 7fa3c6600640 1 -- 172.21.15.46:0/1822145012 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc004900 con 0x560c6b881990 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.561+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa3c00a3c30 0x7fa3c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1822145012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 msgr2=0x560c6b886490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa3c00a3c30 0x7fa3c00a60f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x560c6b8821f0 tx=0x7fa3c800b830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b886490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc0095e0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1822145012 shutdown_connections 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b886b00 0x560c6b888f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b886490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1822145012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b880fa0 0x560c6b8813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1822145012 >> 172.21.15.46:0/1822145012 conn(0x560c6b87c510 msgr2=0x560c6b87c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1822145012 shutdown_connections 2024-04-17T08:00:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1822145012 wait complete. 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.562+0000 7fa3e2d97740 1 -- start start 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 1 Processor -- start 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b881990 0x560c6b88b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b886b00 0x560c6b892970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560c6b889a50 con 0x560c6b880fa0 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560c6b7a0830 con 0x560c6b881990 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560c6b889bd0 con 0x560c6b886b00 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3e2d97740 10 client.0 _ll_register_callbacks cb 0x560c6b76efb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3dcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b881990 0x560c6b88b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36614/0 (socket says 172.21.15.46:36614) 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3dd600640 1 -- 172.21.15.46:0/119106967 learned_addr learned my addr 172.21.15.46:0/119106967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.563+0000 7fa3de000640 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b886b00 0x560c6b892970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3dd600640 1 -- 172.21.15.46:0/119106967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b886b00 msgr2=0x560c6b892970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3dd600640 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b886b00 0x560c6b892970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3dd600640 1 -- 172.21.15.46:0/119106967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b881990 msgr2=0x560c6b88b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3dd600640 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b881990 0x560c6b88b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3dd600640 1 -- 172.21.15.46:0/119106967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c6b7a4920 con 0x560c6b880fa0 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3dd600640 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88b3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc02c240 tx=0x7fa3cc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218092bffa43512a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3baa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3baa00640 1 -- 172.21.15.46:0/119106967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc04d070 con 0x560c6b880fa0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3baa00640 1 -- 172.21.15.46:0/119106967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa3cc03f440 con 0x560c6b880fa0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.564+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc02cb20 con 0x560c6b880fa0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3baa00640 1 -- 172.21.15.46:0/119106967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc058050 con 0x560c6b880fa0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 10 client.15729 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560c6b7727f0 con 0x560c6b880fa0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3baa00640 1 -- 172.21.15.46:0/119106967 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3cc02cb20 con 0x560c6b880fa0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3baa00640 1 -- 172.21.15.46:0/119106967 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fa3cc03fc40 con 0x560c6b880fa0 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 10 client.15729 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 20 client.15729 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b896ad0 0x560c6b898ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 4 client.15729 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.565+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c6b7f4a40 con 0x560c6b896ad0 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.566+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b896ad0 0x560c6b898ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.566+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b896ad0 0x560c6b898ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56454fae67d0717 server_cookie=3bce8cc17bbc7514 in_seq=0 out_seq=0 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.566+0000 7fa3baa00640 10 client.15729 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.567+0000 7fa3baa00640 1 -- 172.21.15.46:0/119106967 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c6b7f4a40 con 0x560c6b896ad0 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.567+0000 7fa3baa00640 10 client.15729 handle_command_reply: tid=0 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.567+0000 7fa3e2d97740 1 client.15729 shutdown 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 msgr2=0x560c6b88b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88b3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc02c240 tx=0x7fa3cc0038a0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 shutdown_connections 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b896ad0 0x560c6b898ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b886b00 0x560c6b892970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b881990 0x560c6b88b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/119106967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88b3f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3baa00640 0 client.15729 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 >> 172.21.15.46:0/119106967 conn(0x560c6b87c510 msgr2=0x560c6b87b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 shutdown_connections 2024-04-17T08:00:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 1 -- 172.21.15.46:0/119106967 wait complete. 2024-04-17T08:00:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.568+0000 7fa3e2d97740 20 client.15729 trim_cache size 0 max 16384 2024-04-17T08:00:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 Processor -- start 2024-04-17T08:00:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 -- start start 2024-04-17T08:00:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b880fa0 0x560c6b87fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b881990 0x560c6b892930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b89e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81af710 con 0x560c6b881990 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81acfd0 con 0x560c6b886b00 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.583+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81aae20 con 0x560c6b880fa0 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b881990 0x560c6b892930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b881990 0x560c6b892930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36616/0 (socket says 172.21.15.46:36616) 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3de000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b89e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 -- 172.21.15.46:0/2905199636 learned_addr learned my addr 172.21.15.46:0/2905199636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dd600640 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b880fa0 0x560c6b87fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 -- 172.21.15.46:0/2905199636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b880fa0 msgr2=0x560c6b87fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b880fa0 0x560c6b87fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 -- 172.21.15.46:0/2905199636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 msgr2=0x560c6b89e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b89e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 -- 172.21.15.46:0/2905199636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c6b4f93f0 con 0x560c6b881990 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3de000640 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b89e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.584+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b881990 0x560c6b892930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa3d819ba30 tx=0x7fa3c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d05e9396a4410c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3c5c00640 1 -- 172.21.15.46:0/2905199636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c802bb50 con 0x560c6b881990 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3c5c00640 1 -- 172.21.15.46:0/2905199636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c8005500 con 0x560c6b881990 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3c5c00640 1 -- 172.21.15.46:0/2905199636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80047a0 con 0x560c6b881990 2024-04-17T08:00:28.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3e2d97740 1 -- 172.21.15.46:0/2905199636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b881990 msgr2=0x560c6b892930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b881990 0x560c6b892930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa3d819ba30 tx=0x7fa3c80046c0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3e2d97740 1 -- 172.21.15.46:0/2905199636 shutdown_connections 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b89e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b881990 0x560c6b892930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/2905199636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b880fa0 0x560c6b87fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.585+0000 7fa3e2d97740 1 -- 172.21.15.46:0/2905199636 >> 172.21.15.46:0/2905199636 conn(0x560c6b87c510 msgr2=0x560c6b87f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.586+0000 7fa3e2d97740 1 -- 172.21.15.46:0/2905199636 shutdown_connections 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.586+0000 7fa3e2d97740 1 -- 172.21.15.46:0/2905199636 wait complete. 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.586+0000 7fa3e2d97740 1 -- start start 2024-04-17T08:00:28.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.586+0000 7fa3e2d97740 1 Processor -- start 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b890030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3e2d97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b891040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81ab490 con 0x560c6b880fa0 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81ac900 con 0x560c6b886b00 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3e2d97740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d81abb60 con 0x560c6b881990 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3e2d97740 10 client.0 _ll_register_callbacks cb 0x7fa3cc004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36624/0 (socket says 172.21.15.46:36624) 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 -- 172.21.15.46:0/1575946535 learned_addr learned my addr 172.21.15.46:0/1575946535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:28.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3de000640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b891040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b890030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 -- 172.21.15.46:0/1575946535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 msgr2=0x560c6b890030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b890030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 -- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 msgr2=0x560c6b891040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b891040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dd600640 1 -- 172.21.15.46:0/1575946535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c6b5c37d0 con 0x560c6b880fa0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b890030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.587+0000 7fa3de000640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b891040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3dd600640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88faf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc02c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e8dcc8fe4e1902 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3baa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3baa00640 1 -- 172.21.15.46:0/1575946535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc04d040 con 0x560c6b880fa0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3baa00640 1 -- 172.21.15.46:0/1575946535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa3cc02cb80 con 0x560c6b880fa0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c8005500 con 0x560c6b880fa0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3baa00640 1 -- 172.21.15.46:0/1575946535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc062050 con 0x560c6b880fa0 2024-04-17T08:00:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3e2d97740 10 client.15735 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.588+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560c6b892800 con 0x560c6b880fa0 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3baa00640 1 -- 172.21.15.46:0/1575946535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c8005500 con 0x560c6b880fa0 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3baa00640 1 -- 172.21.15.46:0/1575946535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7fa3cc0046c0 con 0x560c6b880fa0 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3e2d97740 10 client.15735 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3e2d97740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3e2d97740 20 client.15735 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b8fd2b0 0x560c6b8ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3e2d97740 4 client.15735 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c6b7ee8b0 con 0x560c6b8fd2b0 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.589+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b8fd2b0 0x560c6b8ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3dcc00640 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b8fd2b0 0x560c6b8ff6d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60aad9395e8ca0e0 server_cookie=c099f9d6775e0fe8 in_seq=0 out_seq=0 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3baa00640 10 client.15735 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3baa00640 1 -- 172.21.15.46:0/1575946535 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c6b7ee8b0 con 0x560c6b8fd2b0 2024-04-17T08:00:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3baa00640 10 client.15735 handle_command_reply: tid=0 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 client.15735 shutdown 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 msgr2=0x560c6b88faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88faf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc02c6d0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 shutdown_connections 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x560c6b8fd2b0 0x560c6b8ff6d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560c6b886b00 0x560c6b891040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560c6b881990 0x560c6b890030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 --2- 172.21.15.46:0/1575946535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c6b880fa0 0x560c6b88faf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3baa00640 0 client.15735 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 >> 172.21.15.46:0/1575946535 conn(0x560c6b87c510 msgr2=0x560c6b880000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 shutdown_connections 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.590+0000 7fa3e2d97740 1 -- 172.21.15.46:0/1575946535 wait complete. 2024-04-17T08:00:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e2d97740 20 client.15735 trim_cache size 0 max 16384 2024-04-17T08:00:28.595 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:00:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa3c00a3c30 msgr2=0x7fa3c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa3c00a3c30 0x7fa3c00a60f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x560c6b8821f0 tx=0x7fa3c800b830 comp rx=0 tx=0).stop 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8105d70 msgr2=0x7fa3d81a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8105d70 0x7fa3d81a1ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009fd0 tx=0x7fa3d000ef10 comp rx=0 tx=0).stop 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 shutdown_connections 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa3c00a3c30 0x7fa3c00a60f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3d8105d70 0x7fa3d81a1ad0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3d81053b0 0x7fa3d819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 --2- 172.21.15.46:0/2457797108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3d81041c0 0x7fa3d819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 >> 172.21.15.46:0/2457797108 conn(0x7fa3d80ff750 msgr2=0x7fa3d8108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 shutdown_connections 2024-04-17T08:00:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:28.591+0000 7fa3e0000640 1 -- 172.21.15.46:0/2457797108 wait complete. 2024-04-17T08:00:28.606 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:28 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/639932376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:00:28.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:28 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/639932376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:00:28.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:28 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/639932376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:00:29.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:29 smithi046 ceph-mon[32038]: pgmap v476: 129 pgs: 129 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-04-17T08:00:29.609 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T08:00:29.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:00:29.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T08:00:29.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:29 smithi053 ceph-mon[36437]: pgmap v476: 129 pgs: 129 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-04-17T08:00:29.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:29 smithi062 ceph-mon[36507]: pgmap v476: 129 pgs: 129 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 -- 172.21.15.46:0/2315527044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8100830 msgr2=0x7fede8100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 --2- 172.21.15.46:0/2315527044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8100830 0x7fede8100ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fede0009ca0 tx=0x7fede002c100 comp rx=0 tx=0).stop 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 -- 172.21.15.46:0/2315527044 shutdown_connections 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 --2- 172.21.15.46:0/2315527044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8100830 0x7fede8100ce0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 --2- 172.21.15.46:0/2315527044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede80ffe70 0x7fede81002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 --2- 172.21.15.46:0/2315527044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fede8102380 0x7fede81027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 -- 172.21.15.46:0/2315527044 >> 172.21.15.46:0/2315527044 conn(0x7fede80fb9f0 msgr2=0x7fede80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 -- 172.21.15.46:0/2315527044 shutdown_connections 2024-04-17T08:00:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.853+0000 7fedee200640 1 -- 172.21.15.46:0/2315527044 wait complete. 2024-04-17T08:00:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 Processor -- start 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 -- start start 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fede80ffe70 0x7fede819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede8100830 0x7fede819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8102380 0x7fede81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fede810fca0 con 0x7fede8102380 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fede810fb20 con 0x7fede80ffe70 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.855+0000 7fedee200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fede810fe20 con 0x7fede8100830 2024-04-17T08:00:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7feded800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8102380 0x7fede81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede8100830 0x7fede819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede8100830 0x7fede819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55520/0 (socket says 172.21.15.46:55520) 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 -- 172.21.15.46:0/1238839647 learned_addr learned my addr 172.21.15.46:0/1238839647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fedece00640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fede80ffe70 0x7fede819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 -- 172.21.15.46:0/1238839647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fede80ffe70 msgr2=0x7fede819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fede80ffe70 0x7fede819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 -- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8102380 msgr2=0x7fede81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8102380 0x7fede81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7fede7e00640 1 -- 172.21.15.46:0/1238839647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede81a1ec0 con 0x7fede8100830 2024-04-17T08:00:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.856+0000 7feded800640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8102380 0x7fede81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.857+0000 7fede7e00640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede8100830 0x7fede819a7e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fedd400e9c0 tx=0x7fedd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.857+0000 7fede5600640 1 -- 172.21.15.46:0/1238839647 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd400cd80 con 0x7fede8100830 2024-04-17T08:00:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.857+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81a21b0 con 0x7fede8100830 2024-04-17T08:00:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.857+0000 7fede5600640 1 -- 172.21.15.46:0/1238839647 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd40035f0 con 0x7fede8100830 2024-04-17T08:00:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.857+0000 7fede5600640 1 -- 172.21.15.46:0/1238839647 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4010690 con 0x7fede8100830 2024-04-17T08:00:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.857+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81a2700 con 0x7fede8100830 2024-04-17T08:00:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.860+0000 7fedf256b740 1 Processor -- start 2024-04-17T08:00:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.860+0000 7fedf256b740 1 -- start start 2024-04-17T08:00:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fedf256b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029996c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fedf256b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x560299971590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fedf256b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x560299974090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fedf256b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5602998d1cb0 con 0x560299971c00 2024-04-17T08:00:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fedf256b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560299878370 con 0x56029996c0a0 2024-04-17T08:00:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fedf256b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5602998cdc70 con 0x56029996ca90 2024-04-17T08:00:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029996c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029996c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51698/0 (socket says 172.21.15.46:51698) 2024-04-17T08:00:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 -- 172.21.15.46:0/4120678636 learned_addr learned my addr 172.21.15.46:0/4120678636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fedece00640 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x560299971590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 -- 172.21.15.46:0/4120678636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 msgr2=0x560299971590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x560299971590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 -- 172.21.15.46:0/4120678636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 msgr2=0x560299974090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x560299974090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7feded800640 1 -- 172.21.15.46:0/4120678636 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560299897910 con 0x56029996c0a0 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.861+0000 7fede5600640 1 -- 172.21.15.46:0/1238839647 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fedd40037a0 con 0x7fede8100830 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.862+0000 7feded800640 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029996c4c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fede0009ca0 tx=0x7fede0005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23aa52e277a684db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.862+0000 7fedda000640 1 -- 172.21.15.46:0/4120678636 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0043070 con 0x56029996c0a0 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.862+0000 7fedda000640 1 -- 172.21.15.46:0/4120678636 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0003970 con 0x56029996c0a0 2024-04-17T08:00:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.862+0000 7fedda000640 1 -- 172.21.15.46:0/4120678636 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0039040 con 0x56029996c0a0 2024-04-17T08:00:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 -- 172.21.15.46:0/4120678636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 msgr2=0x56029996c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029996c4c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fede0009ca0 tx=0x7fede0005ef0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 -- 172.21.15.46:0/4120678636 shutdown_connections 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x560299974090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x560299971590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 --2- 172.21.15.46:0/4120678636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029996c4c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 -- 172.21.15.46:0/4120678636 >> 172.21.15.46:0/4120678636 conn(0x560299967610 msgr2=0x560299967a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 -- 172.21.15.46:0/4120678636 shutdown_connections 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fedf256b740 1 -- 172.21.15.46:0/4120678636 wait complete. 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.863+0000 7fede5600640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fedd00a3b40 0x7fedd00a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 -- start start 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fede5600640 1 -- 172.21.15.46:0/1238839647 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fedd40d0aa0 con 0x7fede8100830 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedece00640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fedd00a3b40 0x7fedd00a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 Processor -- start 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996c0a0 0x560299976640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x560299976b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560299971c00 0x56029997dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560299974b50 con 0x56029996ca90 2024-04-17T08:00:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56029988b730 con 0x560299971c00 2024-04-17T08:00:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.864+0000 7fedf256b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560299974cd0 con 0x56029996c0a0 2024-04-17T08:00:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fedd00a3b40 0x7fedd00a6000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x560299968580 tx=0x7feddc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedf256b740 10 client.0 _ll_register_callbacks cb 0x56029988db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996c0a0 0x560299976640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fede7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x560299976b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996c0a0 0x560299976640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55542/0 (socket says 172.21.15.46:55542) 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 -- 172.21.15.46:0/1384138818 learned_addr learned my addr 172.21.15.46:0/1384138818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7feded800640 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560299971c00 0x56029997dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 -- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560299971c00 msgr2=0x56029997dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560299971c00 0x56029997dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 -- 172.21.15.46:0/1384138818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 msgr2=0x560299976b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x560299976b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.865+0000 7fedece00640 1 -- 172.21.15.46:0/1384138818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602996ac7e0 con 0x56029996c0a0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedece00640 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996c0a0 0x560299976640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feddc01e980 tx=0x7feddc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c95102e6a805f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedcea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedcea00640 1 -- 172.21.15.46:0/1384138818 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc02e060 con 0x56029996c0a0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedcea00640 1 -- 172.21.15.46:0/1384138818 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7feddc018030 con 0x56029996c0a0 2024-04-17T08:00:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede0003970 con 0x56029996c0a0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedcea00640 1 -- 172.21.15.46:0/1384138818 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feddc035050 con 0x56029996c0a0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedf256b740 10 client.25876 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56029970cf70 con 0x56029996c0a0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.866+0000 7fedcea00640 1 -- 172.21.15.46:0/1384138818 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0003970 con 0x56029996c0a0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fedcea00640 1 -- 172.21.15.46:0/1384138818 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7feddc01c3a0 con 0x56029996c0a0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fedf256b740 10 client.25876 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fedf256b740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fedf256b740 20 client.25876 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fedf256b740 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999824e0 0x560299984980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fedf256b740 4 client.25876 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602998df6d0 con 0x5602999824e0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.867+0000 7fede7e00640 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999824e0 0x560299984980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.868+0000 7fede7e00640 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999824e0 0x560299984980 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b29584b62465c4 server_cookie=8378ca3cd069bd63 in_seq=0 out_seq=0 2024-04-17T08:00:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.868+0000 7fedcea00640 10 client.25876 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedcea00640 1 -- 172.21.15.46:0/1384138818 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602998df6d0 con 0x5602999824e0 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedcea00640 10 client.25876 handle_command_reply: tid=0 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 client.25876 shutdown 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996c0a0 msgr2=0x560299976640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996c0a0 0x560299976640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feddc01e980 tx=0x7feddc004730 comp rx=0 tx=0).stop 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 shutdown_connections 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999824e0 0x560299984980 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560299971c00 0x56029997dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x560299976b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 --2- 172.21.15.46:0/1384138818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996c0a0 0x560299976640 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 >> 172.21.15.46:0/1384138818 conn(0x560299967610 msgr2=0x560299966990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedcea00640 0 client.25876 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 shutdown_connections 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 1 -- 172.21.15.46:0/1384138818 wait complete. 2024-04-17T08:00:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.869+0000 7fedf256b740 20 client.25876 trim_cache size 0 max 16384 2024-04-17T08:00:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 Processor -- start 2024-04-17T08:00:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 -- start start 2024-04-17T08:00:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x560299976ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x56029997db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x56029996af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fede81af480 con 0x560299971c00 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fede81acd40 con 0x56029996c0a0 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedf256b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fede81aab90 con 0x56029996ca90 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedece00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x560299976ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedece00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x560299976ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51718/0 (socket says 172.21.15.46:51718) 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fedece00640 1 -- 172.21.15.46:0/4255215744 learned_addr learned my addr 172.21.15.46:0/4255215744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7feded800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x56029996af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.890+0000 7fede7e00640 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x56029997db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedece00640 1 -- 172.21.15.46:0/4255215744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 msgr2=0x56029997db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedece00640 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x56029997db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedece00640 1 -- 172.21.15.46:0/4255215744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 msgr2=0x56029996af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedece00640 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x56029996af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedece00640 1 -- 172.21.15.46:0/4255215744 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602995a4170 con 0x56029996c0a0 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7feded800640 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x56029996af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedece00640 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x560299976ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feddc009c00 tx=0x7feddc01ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b5fe5be7c8064c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedd9600640 1 -- 172.21.15.46:0/4255215744 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc009e80 con 0x56029996c0a0 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedd9600640 1 -- 172.21.15.46:0/4255215744 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc01c3a0 con 0x56029996c0a0 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.891+0000 7fedd9600640 1 -- 172.21.15.46:0/4255215744 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc029e10 con 0x56029996c0a0 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 -- 172.21.15.46:0/4255215744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 msgr2=0x560299976ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x560299976ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feddc009c00 tx=0x7feddc01ce00 comp rx=0 tx=0).stop 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 -- 172.21.15.46:0/4255215744 shutdown_connections 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560299971c00 0x56029996af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56029996ca90 0x56029997db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 --2- 172.21.15.46:0/4255215744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x560299976ae0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 -- 172.21.15.46:0/4255215744 >> 172.21.15.46:0/4255215744 conn(0x560299967610 msgr2=0x56029996a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 -- 172.21.15.46:0/4255215744 shutdown_connections 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 -- 172.21.15.46:0/4255215744 wait complete. 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.892+0000 7fedf256b740 1 -- start start 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 1 Processor -- start 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029997b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x56029997bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560299971c00 0x56029997cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fede81ab200 con 0x56029996ca90 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fede81ac670 con 0x56029996c0a0 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fede81ab8d0 con 0x560299971c00 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedf256b740 10 client.0 _ll_register_callbacks cb 0x7feddc009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedece00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029997b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedece00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029997b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51728/0 (socket says 172.21.15.46:51728) 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fedece00640 1 -- 172.21.15.46:0/353932948 learned_addr learned my addr 172.21.15.46:0/353932948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.893+0000 7fede7e00640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x56029997bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7feded800640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560299971c00 0x56029997cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedece00640 1 -- 172.21.15.46:0/353932948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560299971c00 msgr2=0x56029997cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedece00640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560299971c00 0x56029997cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedece00640 1 -- 172.21.15.46:0/353932948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 msgr2=0x56029997bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedece00640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x56029997bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedece00640 1 -- 172.21.15.46:0/353932948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029997d270 con 0x56029996c0a0 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fede7e00640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x56029997bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedece00640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029997b7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feddc01e980 tx=0x7feddc005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245d7c6f59b36daf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedcea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.894+0000 7fedcea00640 1 -- 172.21.15.46:0/353932948 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc01cbc0 con 0x56029996c0a0 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.895+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feddc01c3a0 con 0x56029996c0a0 2024-04-17T08:00:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.895+0000 7fedcea00640 1 -- 172.21.15.46:0/353932948 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7feddc029b90 con 0x56029996c0a0 2024-04-17T08:00:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.895+0000 7fedcea00640 1 -- 172.21.15.46:0/353932948 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feddc052050 con 0x56029996c0a0 2024-04-17T08:00:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.895+0000 7fedf256b740 10 client.25901 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.895+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5602999e8f10 con 0x56029996c0a0 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.895+0000 7fedcea00640 1 -- 172.21.15.46:0/353932948 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feddc01c3a0 con 0x56029996c0a0 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.895+0000 7fedcea00640 1 -- 172.21.15.46:0/353932948 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7feddc004640 con 0x56029996c0a0 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fedf256b740 10 client.25901 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fedf256b740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fedf256b740 20 client.25901 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fedf256b740 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999e97f0 0x5602999ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fedf256b740 4 client.25901 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5602998d9540 con 0x5602999e97f0 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fede7e00640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999e97f0 0x5602999ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fede7e00640 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999e97f0 0x5602999ebc10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb972a01146b9df server_cookie=105db02aac3afe39 in_seq=0 out_seq=0 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.896+0000 7fedcea00640 10 client.25901 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.897+0000 7fedcea00640 1 -- 172.21.15.46:0/353932948 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5602998d9540 con 0x5602999e97f0 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.897+0000 7fedcea00640 10 client.25901 handle_command_reply: tid=0 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.897+0000 7fedf256b740 1 client.25901 shutdown 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 msgr2=0x56029997b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029997b7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feddc01e980 tx=0x7feddc005ab0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 shutdown_connections 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5602999e97f0 0x5602999ebc10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560299971c00 0x56029997cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56029996ca90 0x56029997bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 --2- 172.21.15.46:0/353932948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56029996c0a0 0x56029997b7d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedcea00640 0 client.25901 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 >> 172.21.15.46:0/353932948 conn(0x560299967610 msgr2=0x56029996b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 shutdown_connections 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 1 -- 172.21.15.46:0/353932948 wait complete. 2024-04-17T08:00:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.898+0000 7fedf256b740 20 client.25901 trim_cache size 0 max 16384 2024-04-17T08:00:29.902 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:00:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-17T08:00:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "56e38c36-5170-49a9-8afa-b9a122bb3198", 2024-04-17T08:00:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-17T08:00:29.902 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fedd00a3b40 msgr2=0x7fedd00a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fedd00a3b40 0x7fedd00a6000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x560299968580 tx=0x7feddc005f90 comp rx=0 tx=0).stop 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede8100830 msgr2=0x7fede819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede8100830 0x7fede819a7e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fedd400e9c0 tx=0x7fedd400ee80 comp rx=0 tx=0).stop 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 shutdown_connections 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fedd00a3b40 0x7fedd00a6000 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8102380 0x7fede81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede8100830 0x7fede819a7e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 --2- 172.21.15.46:0/1238839647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fede80ffe70 0x7fede819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 >> 172.21.15.46:0/1238839647 conn(0x7fede80fb9f0 msgr2=0x7fede80fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 shutdown_connections 2024-04-17T08:00:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:29.899+0000 7fedee200640 1 -- 172.21.15.46:0/1238839647 wait complete. 2024-04-17T08:00:29.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:00:29.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:00:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 -- 172.21.15.46:0/814830258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc101000 msgr2=0x7f52fc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 --2- 172.21.15.46:0/814830258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc101000 0x7f52fc101460 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009900 tx=0x7f52e402bc30 comp rx=0 tx=0).stop 2024-04-17T08:00:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 -- 172.21.15.46:0/814830258 shutdown_connections 2024-04-17T08:00:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 --2- 172.21.15.46:0/814830258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 0x7f52fc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 --2- 172.21.15.46:0/814830258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc101000 0x7f52fc101460 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 --2- 172.21.15.46:0/814830258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc1006a0 0x7f52fc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 -- 172.21.15.46:0/814830258 >> 172.21.15.46:0/814830258 conn(0x7f52fc0fc240 msgr2=0x7f52fc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 -- 172.21.15.46:0/814830258 shutdown_connections 2024-04-17T08:00:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.147+0000 7f52fb200640 1 -- 172.21.15.46:0/814830258 wait complete. 2024-04-17T08:00:30.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.148+0000 7f52fb200640 1 Processor -- start 2024-04-17T08:00:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.148+0000 7f52fb200640 1 -- start start 2024-04-17T08:00:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fb200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc1006a0 0x7f52fc19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc101000 0x7f52fc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 0x7f52fc1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc10fc70 con 0x7f52fc101000 2024-04-17T08:00:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fb200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc10faf0 con 0x7f52fc1006a0 2024-04-17T08:00:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc10fdf0 con 0x7f52fc1019a0 2024-04-17T08:00:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52f1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc101000 0x7f52fc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 0x7f52fc1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc1006a0 0x7f52fc19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 0x7f52fc1a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55574/0 (socket says 172.21.15.46:55574) 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 -- 172.21.15.46:0/682688648 learned_addr learned my addr 172.21.15.46:0/682688648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52f1400640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc101000 0x7f52fc19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36676/0 (socket says 172.21.15.46:36676) 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 -- 172.21.15.46:0/682688648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc1006a0 msgr2=0x7f52fc19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc1006a0 0x7f52fc19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 -- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc101000 msgr2=0x7f52fc19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc101000 0x7f52fc19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52fa800640 1 -- 172.21.15.46:0/682688648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc1a1ef0 con 0x7f52fc1019a0 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.149+0000 7f52f9e00640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc1006a0 0x7f52fc19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.150+0000 7f52f1400640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc101000 0x7f52fc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.150+0000 7f52fa800640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 0x7f52fc1a1880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e970 tx=0x7f52f400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.150+0000 7f52f3400640 1 -- 172.21.15.46:0/682688648 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f400ccc0 con 0x7f52fc1019a0 2024-04-17T08:00:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.150+0000 7f52f3400640 1 -- 172.21.15.46:0/682688648 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f400ce60 con 0x7f52fc1019a0 2024-04-17T08:00:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.150+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc1a21e0 con 0x7f52fc1019a0 2024-04-17T08:00:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.150+0000 7f52f3400640 1 -- 172.21.15.46:0/682688648 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4010490 con 0x7f52fc1019a0 2024-04-17T08:00:30.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.150+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc1a2700 con 0x7f52fc1019a0 2024-04-17T08:00:30.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 Processor -- start 2024-04-17T08:00:30.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f52f3400640 1 -- 172.21.15.46:0/682688648 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f52f4005020 con 0x7f52fc1019a0 2024-04-17T08:00:30.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 -- start start 2024-04-17T08:00:30.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d923790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923d60 0x55d33d928860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d92b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d33d88a120 con 0x55d33d923d60 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d33d830340 con 0x55d33d923370 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.154+0000 7f53039af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d33d8860e0 con 0x55d33d928ed0 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d923790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923d60 0x55d33d928860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d923790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51756/0 (socket says 172.21.15.46:51756) 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f1400640 1 -- 172.21.15.46:0/2442157761 learned_addr learned my addr 172.21.15.46:0/2442157761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f1400640 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d92b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f9e00640 1 -- 172.21.15.46:0/2442157761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 msgr2=0x55d33d92b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f9e00640 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d92b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f9e00640 1 -- 172.21.15.46:0/2442157761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 msgr2=0x55d33d923790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f9e00640 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d923790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.155+0000 7f52f9e00640 1 -- 172.21.15.46:0/2442157761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d33d6667d0 con 0x55d33d923d60 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.156+0000 7f52f9e00640 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923d60 0x55d33d928860 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f52ec005fd0 tx=0x7f52ec011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67af6775baa6bb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.156+0000 7f52f3400640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e00a3bb0 0x7f52e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.156+0000 7f52eb000640 1 -- 172.21.15.46:0/2442157761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec003c30 con 0x55d33d923d60 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.156+0000 7f52f3400640 1 -- 172.21.15.46:0/682688648 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f52f40d0bb0 con 0x7f52fc1019a0 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.156+0000 7f52eb000640 1 -- 172.21.15.46:0/2442157761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ec00fe80 con 0x55d33d923d60 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.156+0000 7f52eb000640 1 -- 172.21.15.46:0/2442157761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0193e0 con 0x55d33d923d60 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.156+0000 7f52f1400640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e00a3bb0 0x7f52e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 -- 172.21.15.46:0/2442157761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923d60 msgr2=0x55d33d928860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923d60 0x55d33d928860 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f52ec005fd0 tx=0x7f52ec011c70 comp rx=0 tx=0).stop 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f52f1400640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e00a3bb0 0x7f52e00a6070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55d33d9245c0 tx=0x7f52e4005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 -- 172.21.15.46:0/2442157761 shutdown_connections 2024-04-17T08:00:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d92b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923d60 0x55d33d928860 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 --2- 172.21.15.46:0/2442157761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d923790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 -- 172.21.15.46:0/2442157761 >> 172.21.15.46:0/2442157761 conn(0x55d33d91e8e0 msgr2=0x55d33d91ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 -- 172.21.15.46:0/2442157761 shutdown_connections 2024-04-17T08:00:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.157+0000 7f53039af740 1 -- 172.21.15.46:0/2442157761 wait complete. 2024-04-17T08:00:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.158+0000 7f53039af740 1 -- start start 2024-04-17T08:00:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.159+0000 7f53039af740 1 Processor -- start 2024-04-17T08:00:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f53039af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923370 0x55d33d92d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f53039af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923d60 0x55d33d92ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f53039af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d934e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f53039af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d33d92be80 con 0x55d33d923370 2024-04-17T08:00:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f53039af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d33d843710 con 0x55d33d923d60 2024-04-17T08:00:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f53039af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d33d92c000 con 0x55d33d928ed0 2024-04-17T08:00:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f53039af740 10 client.0 _ll_register_callbacks cb 0x55d33d83eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f52f1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923d60 0x55d33d92ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f52fa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d934e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f52fa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d934e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55584/0 (socket says 172.21.15.46:55584) 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f52f1400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923d60 0x55d33d92ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51770/0 (socket says 172.21.15.46:51770) 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f52fa800640 1 -- 172.21.15.46:0/3403095830 learned_addr learned my addr 172.21.15.46:0/3403095830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.160+0000 7f52f9e00640 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923370 0x55d33d92d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52fa800640 1 -- 172.21.15.46:0/3403095830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923d60 msgr2=0x55d33d92ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52fa800640 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923d60 0x55d33d92ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52fa800640 1 -- 172.21.15.46:0/3403095830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923370 msgr2=0x55d33d92d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52fa800640 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923370 0x55d33d92d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52fa800640 1 -- 172.21.15.46:0/3403095830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33d59c3f0 con 0x55d33d928ed0 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52f9e00640 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923370 0x55d33d92d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52fa800640 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d934e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52f4098f60 tx=0x7f52f409d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47e9a7075a402f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52df400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.161+0000 7f52df400640 1 -- 172.21.15.46:0/3403095830 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40bd070 con 0x55d33d928ed0 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.162+0000 7f52df400640 1 -- 172.21.15.46:0/3403095830 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f52f4005310 con 0x55d33d928ed0 2024-04-17T08:00:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.162+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d33d897b40 con 0x55d33d928ed0 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.162+0000 7f52df400640 1 -- 172.21.15.46:0/3403095830 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52f40a52e0 con 0x55d33d928ed0 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.162+0000 7f53039af740 10 client.25894 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.162+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d33d55c0f0 con 0x55d33d928ed0 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.162+0000 7f52df400640 1 -- 172.21.15.46:0/3403095830 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d33d897b40 con 0x55d33d928ed0 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.162+0000 7f52df400640 1 -- 172.21.15.46:0/3403095830 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f52f409b9c0 con 0x55d33d928ed0 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f53039af740 10 client.25894 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f53039af740 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f53039af740 20 client.25894 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f53039af740 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d941f70 0x55d33d9443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f53039af740 4 client.25894 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d33d8919b0 con 0x55d33d941f70 2024-04-17T08:00:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f52f9e00640 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d941f70 0x55d33d9443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.163+0000 7f52f9e00640 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d941f70 0x55d33d9443e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8253ab177cbecc85 server_cookie=6ce1f89b9b8b2761 in_seq=0 out_seq=0 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.164+0000 7f52df400640 10 client.25894 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f52df400640 1 -- 172.21.15.46:0/3403095830 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d33d8919b0 con 0x55d33d941f70 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f52df400640 10 client.25894 handle_command_reply: tid=0 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 client.25894 shutdown 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 msgr2=0x55d33d934e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d934e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52f4098f60 tx=0x7f52f409d340 comp rx=0 tx=0).stop 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 shutdown_connections 2024-04-17T08:00:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d941f70 0x55d33d9443e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d928ed0 0x55d33d934e10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923d60 0x55d33d92ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 --2- 172.21.15.46:0/3403095830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d923370 0x55d33d92d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f52df400640 0 client.25894 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.165+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 >> 172.21.15.46:0/3403095830 conn(0x55d33d91e8e0 msgr2=0x55d33d91dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.166+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 shutdown_connections 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.166+0000 7f53039af740 1 -- 172.21.15.46:0/3403095830 wait complete. 2024-04-17T08:00:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.166+0000 7f53039af740 20 client.25894 trim_cache size 0 max 16384 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.187+0000 7f53039af740 1 Processor -- start 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f53039af740 1 -- start start 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f53039af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d925300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f53039af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d934dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f53039af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d91d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f53039af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1ab180 con 0x55d33d928ed0 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f53039af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1a8a40 con 0x55d33d923370 2024-04-17T08:00:30.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f53039af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1a6890 con 0x55d33d923d60 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d925300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d925300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51772/0 (socket says 172.21.15.46:51772) 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f52f9e00640 1 -- 172.21.15.46:0/1238053053 learned_addr learned my addr 172.21.15.46:0/1238053053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f52fa800640 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d91d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.188+0000 7f52f1400640 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d934dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52f9e00640 1 -- 172.21.15.46:0/1238053053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 msgr2=0x55d33d934dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52f9e00640 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d934dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52f9e00640 1 -- 172.21.15.46:0/1238053053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 msgr2=0x55d33d91d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52f9e00640 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d91d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52f9e00640 1 -- 172.21.15.46:0/1238053053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d33d9245f0 con 0x55d33d923370 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52fa800640 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d91d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52f9e00640 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d925300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52ec011730 tx=0x7f52ec009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e43df32d2d5c0e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52ea600640 1 -- 172.21.15.46:0/1238053053 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec030070 con 0x55d33d923370 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52ea600640 1 -- 172.21.15.46:0/1238053053 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ec00fba0 con 0x55d33d923370 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.189+0000 7f52ea600640 1 -- 172.21.15.46:0/1238053053 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec01f040 con 0x55d33d923370 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 -- 172.21.15.46:0/1238053053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 msgr2=0x55d33d925300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d925300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52ec011730 tx=0x7f52ec009fb0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 -- 172.21.15.46:0/1238053053 shutdown_connections 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d91d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d934dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 --2- 172.21.15.46:0/1238053053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d925300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 -- 172.21.15.46:0/1238053053 >> 172.21.15.46:0/1238053053 conn(0x55d33d91e8e0 msgr2=0x55d33d9223d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 -- 172.21.15.46:0/1238053053 shutdown_connections 2024-04-17T08:00:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.190+0000 7f53039af740 1 -- 172.21.15.46:0/1238053053 wait complete. 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 -- start start 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 Processor -- start 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d932780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d932cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d933bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1a6f00 con 0x55d33d928ed0 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1a8370 con 0x55d33d923370 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc1a75d0 con 0x55d33d923d60 2024-04-17T08:00:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.191+0000 7f53039af740 10 client.0 _ll_register_callbacks cb 0x7f52f4004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d932780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d932780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51786/0 (socket says 172.21.15.46:51786) 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d932cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 -- 172.21.15.46:0/1063861056 learned_addr learned my addr 172.21.15.46:0/1063861056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52fa800640 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d933bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 -- 172.21.15.46:0/1063861056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 msgr2=0x55d33d932cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d932cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 -- 172.21.15.46:0/1063861056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 msgr2=0x55d33d933bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d933bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 -- 172.21.15.46:0/1063861056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33d9341e0 con 0x55d33d923370 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52fa800640 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d933bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52f9e00640 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d932780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52ec006010 tx=0x7f52ec00f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1229aa20924ef6d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.192+0000 7f52df400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f52df400640 1 -- 172.21.15.46:0/1063861056 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec038060 con 0x55d33d923370 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f52df400640 1 -- 172.21.15.46:0/1063861056 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f52ec0043d0 con 0x55d33d923370 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ec00fba0 con 0x55d33d923370 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f52df400640 1 -- 172.21.15.46:0/1063861056 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52ec030070 con 0x55d33d923370 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 10 client.25919 fetch_fsmap learned FSMap version 24 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d33d9a8640 con 0x55d33d923370 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f52df400640 1 -- 172.21.15.46:0/1063861056 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ec00fba0 con 0x55d33d923370 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f52df400640 1 -- 172.21.15.46:0/1063861056 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 24) ==== 8347+0+0 (secure 0 0 0) 0x7f52ec018070 con 0x55d33d923370 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 10 client.25919 fetch_fsmap finished waiting for FSMap version 24 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 20 client.25919 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d9a8da0 0x55d33d9ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 4 client.25919 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.193+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d33d80d420 con 0x55d33d9a8da0 2024-04-17T08:00:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.194+0000 7f52f1400640 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d9a8da0 0x55d33d9ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.194+0000 7f52f1400640 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d9a8da0 0x55d33d9ab1c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f283854f67418c5 server_cookie=cac6bb474f4ad802 in_seq=0 out_seq=0 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.194+0000 7f52df400640 10 client.25919 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.195+0000 7f52df400640 1 -- 172.21.15.46:0/1063861056 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d33d80d420 con 0x55d33d9a8da0 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.195+0000 7f52df400640 10 client.25919 handle_command_reply: tid=0 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.195+0000 7f53039af740 1 client.25919 shutdown 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.195+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 msgr2=0x55d33d932780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.195+0000 7f53039af740 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d932780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52ec006010 tx=0x7f52ec00f960 comp rx=0 tx=0).stop 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 shutdown_connections 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55d33d9a8da0 0x55d33d9ab1c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d33d928ed0 0x55d33d933bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d33d923d60 0x55d33d932cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f52df400640 0 client.25919 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 --2- 172.21.15.46:0/1063861056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d33d923370 0x55d33d932780 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 >> 172.21.15.46:0/1063861056 conn(0x55d33d91e8e0 msgr2=0x55d33d92c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 shutdown_connections 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 1 -- 172.21.15.46:0/1063861056 wait complete. 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.196+0000 7f53039af740 20 client.25919 trim_cache size 0 max 16384 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-17T08:00:30.199 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e00a3bb0 msgr2=0x7f52e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e00a3bb0 0x7f52e00a6070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55d33d9245c0 tx=0x7f52e4005b50 comp rx=0 tx=0).stop 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 msgr2=0x7f52fc1a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 0x7f52fc1a1880 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e970 tx=0x7f52f400ed70 comp rx=0 tx=0).stop 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 shutdown_connections 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f52e00a3bb0 0x7f52e00a6070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52fc1019a0 0x7f52fc1a1880 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc101000 0x7f52fc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 --2- 172.21.15.46:0/682688648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52fc1006a0 0x7f52fc19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 >> 172.21.15.46:0/682688648 conn(0x7f52fc0fc240 msgr2=0x7f52fc0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 shutdown_connections 2024-04-17T08:00:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.197+0000 7f52fb200640 1 -- 172.21.15.46:0/682688648 wait complete. 2024-04-17T08:00:30.214 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T08:00:30.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T08:00:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 -- 172.21.15.46:0/2521394662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 msgr2=0x7f3e581022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 --2- 172.21.15.46:0/2521394662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e581022b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009900 tx=0x7f3e4c02bc30 comp rx=0 tx=0).stop 2024-04-17T08:00:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 -- 172.21.15.46:0/2521394662 shutdown_connections 2024-04-17T08:00:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 --2- 172.21.15.46:0/2521394662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e581039a0 0x7f3e5810a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 --2- 172.21.15.46:0/2521394662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e58102fe0 0x7f3e58103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 --2- 172.21.15.46:0/2521394662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e581022b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 -- 172.21.15.46:0/2521394662 >> 172.21.15.46:0/2521394662 conn(0x7f3e580fd420 msgr2=0x7f3e580ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 -- 172.21.15.46:0/2521394662 shutdown_connections 2024-04-17T08:00:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.444+0000 7f3e5e600640 1 -- 172.21.15.46:0/2521394662 wait complete. 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.445+0000 7f3e5e600640 1 Processor -- start 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.445+0000 7f3e5e600640 1 -- start start 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5e600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e5819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e58102fe0 0x7f3e5819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e581039a0 0x7f3e581a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5810db60 con 0x7f3e58102fe0 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5e600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5810d9e0 con 0x7f3e58101e90 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5810dce0 con 0x7f3e581039a0 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e5819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e58102fe0 0x7f3e5819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5d200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e5819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51802/0 (socket says 172.21.15.46:51802) 2024-04-17T08:00:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5d200640 1 -- 172.21.15.46:0/87593383 learned_addr learned my addr 172.21.15.46:0/87593383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.446+0000 7f3e5dc00640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e581039a0 0x7f3e581a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5d200640 1 -- 172.21.15.46:0/87593383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e581039a0 msgr2=0x7f3e581a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5d200640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e581039a0 0x7f3e581a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5d200640 1 -- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e58102fe0 msgr2=0x7f3e5819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5d200640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e58102fe0 0x7f3e5819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5d200640 1 -- 172.21.15.46:0/87593383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e581a1fb0 con 0x7f3e58101e90 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5dc00640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e581039a0 0x7f3e581a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5d200640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e5819a390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c000c00 tx=0x7f3e4c005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e55600640 1 -- 172.21.15.46:0/87593383 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c043020 con 0x7f3e58101e90 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e55600640 1 -- 172.21.15.46:0/87593383 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e4c0054e0 con 0x7f3e58101e90 2024-04-17T08:00:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e581a2240 con 0x7f3e58101e90 2024-04-17T08:00:30.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e55600640 1 -- 172.21.15.46:0/87593383 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c003e60 con 0x7f3e58101e90 2024-04-17T08:00:30.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.447+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e581a27e0 con 0x7f3e58101e90 2024-04-17T08:00:30.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.449+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e581a29e0 con 0x7f3e58101e90 2024-04-17T08:00:30.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.451+0000 7f3e55600640 1 -- 172.21.15.46:0/87593383 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e4c046020 con 0x7f3e58101e90 2024-04-17T08:00:30.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.452+0000 7f3e55600640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3e400a3c20 0x7f3e400a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:30.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.452+0000 7f3e55600640 1 -- 172.21.15.46:0/87593383 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f3e4c0f29f0 con 0x7f3e58101e90 2024-04-17T08:00:30.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.452+0000 7f3e57e00640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3e400a3c20 0x7f3e400a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:30.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.453+0000 7f3e57e00640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3e400a3c20 0x7f3e400a60e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3e5819b8a0 tx=0x7f3e4400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.456+0000 7f3e55600640 1 -- 172.21.15.46:0/87593383 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e4c04d020 con 0x7f3e58101e90 2024-04-17T08:00:30.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:30 smithi046 ceph-mon[32038]: pgmap v477: 129 pgs: 129 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-17T08:00:30.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:30 smithi046 ceph-mon[32038]: scrub queued for path: / 2024-04-17T08:00:30.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:30 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:30.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:30 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:00:30.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:30 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:30.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:30 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:00:30.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:30 smithi053 ceph-mon[36437]: pgmap v477: 129 pgs: 129 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-17T08:00:30.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:30 smithi053 ceph-mon[36437]: scrub queued for path: / 2024-04-17T08:00:30.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:30 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:30.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:30 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:00:30.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:30 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:30.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:30 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:00:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.737+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e581a2bb0 con 0x7f3e58101e90 2024-04-17T08:00:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.738+0000 7f3e55600640 1 -- 172.21.15.46:0/87593383 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11870 (secure 0 0 0) 0x7f3e4c0befe0 con 0x7f3e58101e90 2024-04-17T08:00:30.739 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:00:30.739 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T08:00:30.055240+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24556,24542]},"id":1}]} 2024-04-17T08:00:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-17T08:00:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3e400a3c20 msgr2=0x7f3e400a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3e400a3c20 0x7f3e400a60e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3e5819b8a0 tx=0x7f3e4400b390 comp rx=0 tx=0).stop 2024-04-17T08:00:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 msgr2=0x7f3e5819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e5819a390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c000c00 tx=0x7f3e4c005dd0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 shutdown_connections 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3e400a3c20 0x7f3e400a60e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e581039a0 0x7f3e581a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e58102fe0 0x7f3e5819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.741+0000 7f3e5e600640 1 --2- 172.21.15.46:0/87593383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e58101e90 0x7f3e5819a390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.742+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 >> 172.21.15.46:0/87593383 conn(0x7f3e580fd420 msgr2=0x7f3e58074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.742+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 shutdown_connections 2024-04-17T08:00:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:30.742+0000 7f3e5e600640 1 -- 172.21.15.46:0/87593383 wait complete. 2024-04-17T08:00:30.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T08:00:30.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:30 smithi062 ceph-mon[36507]: pgmap v477: 129 pgs: 129 active+clean; 14 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-17T08:00:30.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:30 smithi062 ceph-mon[36507]: scrub queued for path: / 2024-04-17T08:00:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:30 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:30 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:00:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:30 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:30.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:30 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.039+0000 7f356a400640 1 -- 172.21.15.46:0/145387602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564101b70 msgr2=0x7f3564101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.039+0000 7f356a400640 1 --2- 172.21.15.46:0/145387602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564101b70 0x7f3564101ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f355c009ca0 tx=0x7f355c02c100 comp rx=0 tx=0).stop 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.040+0000 7f356a400640 1 -- 172.21.15.46:0/145387602 shutdown_connections 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.040+0000 7f356a400640 1 --2- 172.21.15.46:0/145387602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564101b70 0x7f3564101ff0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.040+0000 7f356a400640 1 --2- 172.21.15.46:0/145387602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35641011d0 0x7f3564101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.040+0000 7f356a400640 1 --2- 172.21.15.46:0/145387602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3564100870 0x7f3564100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.040+0000 7f356a400640 1 -- 172.21.15.46:0/145387602 >> 172.21.15.46:0/145387602 conn(0x7f35640fc430 msgr2=0x7f35640fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.040+0000 7f356a400640 1 -- 172.21.15.46:0/145387602 shutdown_connections 2024-04-17T08:00:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.040+0000 7f356a400640 1 -- 172.21.15.46:0/145387602 wait complete. 2024-04-17T08:00:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 Processor -- start 2024-04-17T08:00:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 -- start start 2024-04-17T08:00:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564100870 0x7f356419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35641011d0 0x7f356419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3564101b70 0x7f35641a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f356410fe40 con 0x7f35641011d0 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f356410fcc0 con 0x7f3564100870 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f356a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f356410ffc0 con 0x7f3564101b70 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f3563400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564100870 0x7f356419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f3562a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35641011d0 0x7f356419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f3563400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564100870 0x7f356419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51830/0 (socket says 172.21.15.46:51830) 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.041+0000 7f3563400640 1 -- 172.21.15.46:0/63359510 learned_addr learned my addr 172.21.15.46:0/63359510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f3563e00640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3564101b70 0x7f35641a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f3563400640 1 -- 172.21.15.46:0/63359510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3564101b70 msgr2=0x7f35641a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f3563400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3564101b70 0x7f35641a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f3563400640 1 -- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35641011d0 msgr2=0x7f356419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f3563400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35641011d0 0x7f356419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f3563400640 1 -- 172.21.15.46:0/63359510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35641a2070 con 0x7f3564100870 2024-04-17T08:00:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f3563400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564100870 0x7f356419a450 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f355400e9f0 tx=0x7f355400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f355be00640 1 -- 172.21.15.46:0/63359510 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400cd80 con 0x7f3564100870 2024-04-17T08:00:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f355be00640 1 -- 172.21.15.46:0/63359510 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3554010910 con 0x7f3564100870 2024-04-17T08:00:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f355be00640 1 -- 172.21.15.46:0/63359510 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554018990 con 0x7f3564100870 2024-04-17T08:00:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.042+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35641a2360 con 0x7f3564100870 2024-04-17T08:00:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.043+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35641a2900 con 0x7f3564100870 2024-04-17T08:00:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.046+0000 7f355be00640 1 -- 172.21.15.46:0/63359510 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3554010ac0 con 0x7f3564100870 2024-04-17T08:00:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.046+0000 7f356d1c8740 1 Processor -- start 2024-04-17T08:00:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.046+0000 7f356d1c8740 1 -- start start 2024-04-17T08:00:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30830c990 0x55b308311490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b308311b00 0x55b308313f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f356d1c8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b308272020 con 0x55b30830bfa0 2024-04-17T08:00:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f356d1c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b308218460 con 0x55b308311b00 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f356d1c8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b30826dfe0 con 0x55b30830c990 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3562a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b308311b00 0x55b308313f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3563e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3562a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b308311b00 0x55b308313f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51844/0 (socket says 172.21.15.46:51844) 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3562a00640 1 -- 172.21.15.46:0/2857579287 learned_addr learned my addr 172.21.15.46:0/2857579287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3563400640 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30830c990 0x55b308311490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3562a00640 1 -- 172.21.15.46:0/2857579287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30830c990 msgr2=0x55b308311490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3562a00640 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30830c990 0x55b308311490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3562a00640 1 -- 172.21.15.46:0/2857579287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 msgr2=0x55b30830c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.047+0000 7f3562a00640 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f3562a00640 1 -- 172.21.15.46:0/2857579287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b308220040 con 0x55b308311b00 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f3563400640 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30830c990 0x55b308311490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f355be00640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35480a3c30 0x7f35480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f3562a00640 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b308311b00 0x55b308313f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f355000bc90 tx=0x7f35500107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce0b5b115f02226 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f355be00640 1 -- 172.21.15.46:0/63359510 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f35540d0a60 con 0x7f3564100870 2024-04-17T08:00:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f3558c00640 1 -- 172.21.15.46:0/2857579287 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550005220 con 0x55b308311b00 2024-04-17T08:00:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f3563e00640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35480a3c30 0x7f35480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f3558c00640 1 -- 172.21.15.46:0/2857579287 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355000be40 con 0x55b308311b00 2024-04-17T08:00:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.048+0000 7f3558c00640 1 -- 172.21.15.46:0/2857579287 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550011770 con 0x55b308311b00 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f3563e00640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35480a3c30 0x7f35480a60f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f355c02c640 tx=0x7f355c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 -- 172.21.15.46:0/2857579287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b308311b00 msgr2=0x55b308313f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b308311b00 0x55b308313f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f355000bc90 tx=0x7f35500107a0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 -- 172.21.15.46:0/2857579287 shutdown_connections 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b308311b00 0x55b308313f90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30830c990 0x55b308311490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 --2- 172.21.15.46:0/2857579287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 -- 172.21.15.46:0/2857579287 >> 172.21.15.46:0/2857579287 conn(0x55b308307510 msgr2=0x55b308307910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 -- 172.21.15.46:0/2857579287 shutdown_connections 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.049+0000 7f356d1c8740 1 -- 172.21.15.46:0/2857579287 wait complete. 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.050+0000 7f356d1c8740 1 -- start start 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.050+0000 7f356d1c8740 1 Processor -- start 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830bfa0 0x55b3083164e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830c990 0x55b308316a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f356d1c8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b308314ab0 con 0x55b30830c990 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f356d1c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b30822b830 con 0x55b30830bfa0 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f356d1c8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b308314c30 con 0x55b308311b00 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f356d1c8740 10 client.0 _ll_register_callbacks cb 0x55b3081f9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830bfa0 0x55b3083164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830bfa0 0x55b3083164e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51858/0 (socket says 172.21.15.46:51858) 2024-04-17T08:00:31.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 -- 172.21.15.46:0/3354908635 learned_addr learned my addr 172.21.15.46:0/3354908635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3562a00640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830c990 0x55b308316a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563e00640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 -- 172.21.15.46:0/3354908635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 msgr2=0x55b30831da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 -- 172.21.15.46:0/3354908635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830c990 msgr2=0x55b308316a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830c990 0x55b308316a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563400640 1 -- 172.21.15.46:0/3354908635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b30822f920 con 0x55b30830bfa0 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3562a00640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830c990 0x55b308316a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.051+0000 7f3563e00640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f3563400640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830bfa0 0x55b3083164e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f355400e9c0 tx=0x7f355409c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4389908eeecddd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f354d600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f354d600640 1 -- 172.21.15.46:0/3354908635 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554018d30 con 0x55b30830bfa0 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f354d600640 1 -- 172.21.15.46:0/3354908635 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f355409be50 con 0x55b30830bfa0 2024-04-17T08:00:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b30827fa40 con 0x55b30830bfa0 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f354d600640 1 -- 172.21.15.46:0/3354908635 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35540a0d00 con 0x55b30830bfa0 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f356d1c8740 10 client.25937 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3081fd7f0 con 0x55b30830bfa0 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f354d600640 1 -- 172.21.15.46:0/3354908635 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b30827fa40 con 0x55b30830bfa0 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f354d600640 1 -- 172.21.15.46:0/3354908635 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f35540bbb10 con 0x55b30830bfa0 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f356d1c8740 10 client.25937 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.052+0000 7f356d1c8740 10 client.25937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.053+0000 7f356d1c8740 20 client.25937 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.053+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308321f60 0x55b3083243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.053+0000 7f356d1c8740 4 client.25937 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.053+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3082798b0 con 0x55b308321f60 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.053+0000 7f3562a00640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308321f60 0x55b3083243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.053+0000 7f3562a00640 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308321f60 0x55b3083243d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d696afbaa37ea803 server_cookie=bed9c9709d0bde9f in_seq=0 out_seq=0 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.053+0000 7f354d600640 10 client.25937 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.054+0000 7f354d600640 1 -- 172.21.15.46:0/3354908635 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3082798b0 con 0x55b308321f60 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.054+0000 7f354d600640 10 client.25937 handle_command_reply: tid=0 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.054+0000 7f356d1c8740 1 client.25937 shutdown 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.054+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830bfa0 msgr2=0x55b3083164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.054+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830bfa0 0x55b3083164e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f355400e9c0 tx=0x7f355409c8c0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 shutdown_connections 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308321f60 0x55b3083243d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830c990 0x55b308316a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3354908635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830bfa0 0x55b3083164e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f354d600640 0 client.25937 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 >> 172.21.15.46:0/3354908635 conn(0x55b308307510 msgr2=0x55b308306710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 shutdown_connections 2024-04-17T08:00:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 1 -- 172.21.15.46:0/3354908635 wait complete. 2024-04-17T08:00:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.055+0000 7f356d1c8740 20 client.25937 trim_cache size 0 max 16384 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 Processor -- start 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 -- start start 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b308315b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35641af6d0 con 0x55b30830bfa0 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35641acf90 con 0x55b30830c990 2024-04-17T08:00:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f356d1c8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35641aade0 con 0x55b308311b00 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f3563400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.070+0000 7f3562a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3563400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36790/0 (socket says 172.21.15.46:36790) 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3563400640 1 -- 172.21.15.46:0/36099764 learned_addr learned my addr 172.21.15.46:0/36099764 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3562a00640 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51866/0 (socket says 172.21.15.46:51866) 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3563e00640 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b308315b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3562a00640 1 -- 172.21.15.46:0/36099764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 msgr2=0x55b308315b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3562a00640 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b308315b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3562a00640 1 -- 172.21.15.46:0/36099764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 msgr2=0x55b30830ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3562a00640 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3562a00640 1 -- 172.21.15.46:0/36099764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b307f843f0 con 0x55b30830c990 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3563400640 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f3562a00640 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831da20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f355000bc60 tx=0x7f355000b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6dbd4e0d97d74cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.071+0000 7f354fe00640 1 -- 172.21.15.46:0/36099764 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500055a0 con 0x55b30830c990 2024-04-17T08:00:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f354fe00640 1 -- 172.21.15.46:0/36099764 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3550029170 con 0x55b30830c990 2024-04-17T08:00:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f354fe00640 1 -- 172.21.15.46:0/36099764 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550003850 con 0x55b30830c990 2024-04-17T08:00:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f356d1c8740 1 -- 172.21.15.46:0/36099764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 msgr2=0x55b30831da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f356d1c8740 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831da20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f355000bc60 tx=0x7f355000b630 comp rx=0 tx=0).stop 2024-04-17T08:00:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f356d1c8740 1 -- 172.21.15.46:0/36099764 shutdown_connections 2024-04-17T08:00:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f356d1c8740 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b308315b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f356d1c8740 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831da20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f356d1c8740 1 --2- 172.21.15.46:0/36099764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30830ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.072+0000 7f356d1c8740 1 -- 172.21.15.46:0/36099764 >> 172.21.15.46:0/36099764 conn(0x55b308307510 msgr2=0x55b30830a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.073+0000 7f356d1c8740 1 -- 172.21.15.46:0/36099764 shutdown_connections 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.073+0000 7f356d1c8740 1 -- 172.21.15.46:0/36099764 wait complete. 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.073+0000 7f356d1c8740 1 -- start start 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.073+0000 7f356d1c8740 1 Processor -- start 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30831ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f356d1c8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f356d1c8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35641ab450 con 0x55b30830bfa0 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f356d1c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35641ac8c0 con 0x55b30830c990 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f356d1c8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35641abb20 con 0x55b308311b00 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f356d1c8740 10 client.0 _ll_register_callbacks cb 0x7f3554098d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3563400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30831ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51878/0 (socket says 172.21.15.46:51878) 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 -- 172.21.15.46:0/3389066591 learned_addr learned my addr 172.21.15.46:0/3389066591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3563e00640 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 -- 172.21.15.46:0/3389066591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 msgr2=0x55b30831c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 -- 172.21.15.46:0/3389066591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 msgr2=0x55b30831ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30831ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3562a00640 1 -- 172.21.15.46:0/3389066591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b30804e7d0 con 0x55b30830c990 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.074+0000 7f3563400640 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30831ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f3562a00640 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831b280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f355000b6c0 tx=0x7f3550005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3952ed1e0150262 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f354d600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f354d600640 1 -- 172.21.15.46:0/3389066591 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355002f040 con 0x55b30830c990 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f354d600640 1 -- 172.21.15.46:0/3389066591 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f3550004020 con 0x55b30830c990 2024-04-17T08:00:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3550029170 con 0x55b30830c990 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f354d600640 1 -- 172.21.15.46:0/3389066591 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35500055a0 con 0x55b30830c990 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f356d1c8740 10 client.25943 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b30831d9b0 con 0x55b30830c990 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.075+0000 7f354d600640 1 -- 172.21.15.46:0/3389066591 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3550029170 con 0x55b30830c990 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f354d600640 1 -- 172.21.15.46:0/3389066591 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f3550032070 con 0x55b30830c990 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f356d1c8740 10 client.25943 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f356d1c8740 10 client.25943 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f356d1c8740 20 client.25943 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308388710 0x55b30838ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f356d1c8740 4 client.25943 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3081f5570 con 0x55b308388710 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.076+0000 7f3563400640 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308388710 0x55b30838ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f3563400640 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308388710 0x55b30838ab00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9273c9c595f09f9 server_cookie=4cce945969a203c1 in_seq=0 out_seq=0 2024-04-17T08:00:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f354d600640 10 client.25943 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f354d600640 1 -- 172.21.15.46:0/3389066591 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3081f5570 con 0x55b308388710 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f354d600640 10 client.25943 handle_command_reply: tid=0 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 client.25943 shutdown 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 msgr2=0x55b30831b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831b280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f355000b6c0 tx=0x7f3550005f70 comp rx=0 tx=0).stop 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 shutdown_connections 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55b308388710 0x55b30838ab00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b308311b00 0x55b30831c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b30830c990 0x55b30831b280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f354d600640 0 client.25943 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 --2- 172.21.15.46:0/3389066591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b30830bfa0 0x55b30831ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 >> 172.21.15.46:0/3389066591 conn(0x55b308307510 msgr2=0x55b30830b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 shutdown_connections 2024-04-17T08:00:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.077+0000 7f356d1c8740 1 -- 172.21.15.46:0/3389066591 wait complete. 2024-04-17T08:00:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356d1c8740 20 client.25943 trim_cache size 0 max 16384 2024-04-17T08:00:31.082 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35480a3c30 msgr2=0x7f35480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35480a3c30 0x7f35480a60f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f355c02c640 tx=0x7f355c032590 comp rx=0 tx=0).stop 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564100870 msgr2=0x7f356419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564100870 0x7f356419a450 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f355400e9f0 tx=0x7f355400eeb0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 shutdown_connections 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35480a3c30 0x7f35480a60f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3564101b70 0x7f35641a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35641011d0 0x7f356419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 --2- 172.21.15.46:0/63359510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3564100870 0x7f356419a450 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.078+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 >> 172.21.15.46:0/63359510 conn(0x7f35640fc430 msgr2=0x7f35640fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.079+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 shutdown_connections 2024-04-17T08:00:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.079+0000 7f356a400640 1 -- 172.21.15.46:0/63359510 wait complete. 2024-04-17T08:00:31.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T08:00:31.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:31 smithi046 ceph-mon[32038]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:00:31.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:31 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/87593383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.371+0000 7f520ea00640 1 -- 172.21.15.46:0/1487808415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 msgr2=0x7f5208078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.371+0000 7f520ea00640 1 --2- 172.21.15.46:0/1487808415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 0x7f5208078da0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5200009ca0 tx=0x7f520002c100 comp rx=0 tx=0).stop 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.372+0000 7f520ea00640 1 -- 172.21.15.46:0/1487808415 shutdown_connections 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.372+0000 7f520ea00640 1 --2- 172.21.15.46:0/1487808415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 0x7f5208078da0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.372+0000 7f520ea00640 1 --2- 172.21.15.46:0/1487808415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208077b60 0x7f5208077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.372+0000 7f520ea00640 1 --2- 172.21.15.46:0/1487808415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5208076920 0x7f5208076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.372+0000 7f520ea00640 1 -- 172.21.15.46:0/1487808415 >> 172.21.15.46:0/1487808415 conn(0x7f52080ff570 msgr2=0x7f52081019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.372+0000 7f520ea00640 1 -- 172.21.15.46:0/1487808415 shutdown_connections 2024-04-17T08:00:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.372+0000 7f520ea00640 1 -- 172.21.15.46:0/1487808415 wait complete. 2024-04-17T08:00:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 Processor -- start 2024-04-17T08:00:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 -- start start 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208076920 0x7f520819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5208077b60 0x7f520819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 0x7f52081a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5208114090 con 0x7f5208077b60 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5208113f10 con 0x7f5208078500 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.373+0000 7f520ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5208114210 con 0x7f5208076920 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208076920 0x7f520819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5208077b60 0x7f520819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208076920 0x7f520819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55686/0 (socket says 172.21.15.46:55686) 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 -- 172.21.15.46:0/3997593995 learned_addr learned my addr 172.21.15.46:0/3997593995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520e000640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 0x7f52081a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 -- 172.21.15.46:0/3997593995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 msgr2=0x7f52081a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 0x7f52081a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 -- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5208077b60 msgr2=0x7f520819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5208077b60 0x7f520819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 -- 172.21.15.46:0/3997593995 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52081a6280 con 0x7f5208076920 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520e000640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 0x7f52081a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.374+0000 7f520d600640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208076920 0x7f520819e660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9f0 tx=0x7f51fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.375+0000 7f5206000640 1 -- 172.21.15.46:0/3997593995 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00cd80 con 0x7f5208076920 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.375+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52081a6570 con 0x7f5208076920 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.375+0000 7f5206000640 1 -- 172.21.15.46:0/3997593995 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc010910 con 0x7f5208076920 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.375+0000 7f5206000640 1 -- 172.21.15.46:0/3997593995 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0188f0 con 0x7f5208076920 2024-04-17T08:00:31.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.375+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52081a6ac0 con 0x7f5208076920 2024-04-17T08:00:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.378+0000 7f5206000640 1 -- 172.21.15.46:0/3997593995 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51fc010ac0 con 0x7f5208076920 2024-04-17T08:00:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.378+0000 7f5212d82740 1 Processor -- start 2024-04-17T08:00:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f5212d82740 1 -- start start 2024-04-17T08:00:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f5212d82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f55f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f5212d82740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f564650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f5212d82740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f567150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f5212d82740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56059f4c5020 con 0x56059f564cc0 2024-04-17T08:00:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f5212d82740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56059f46b460 con 0x56059f55fb50 2024-04-17T08:00:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f5212d82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56059f4c0fe0 con 0x56059f55f160 2024-04-17T08:00:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f520d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f564650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f520e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f55f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f520cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f567150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f520d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f564650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51896/0 (socket says 172.21.15.46:51896) 2024-04-17T08:00:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f520e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f55f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55698/0 (socket says 172.21.15.46:55698) 2024-04-17T08:00:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.379+0000 7f520d600640 1 -- 172.21.15.46:0/2728992528 learned_addr learned my addr 172.21.15.46:0/2728992528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f520d600640 1 -- 172.21.15.46:0/2728992528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 msgr2=0x56059f55f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f520d600640 1 --2- 172.21.15.46:0/2728992528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f55f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f520d600640 1 -- 172.21.15.46:0/2728992528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 msgr2=0x56059f567150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f520d600640 1 --2- 172.21.15.46:0/2728992528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f567150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f5206000640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f51f00a3bb0 0x7f51f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f520d600640 1 -- 172.21.15.46:0/2728992528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059f473040 con 0x56059f55fb50 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f5206000640 1 -- 172.21.15.46:0/3997593995 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f51fc0d0b80 con 0x7f5208076920 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f520d600640 1 --2- 172.21.15.46:0/2728992528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f564650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51fc09c820 tx=0x7f51fc0d95d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb49c06ac0dc4b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f520e000640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f51f00a3bb0 0x7f51f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f51fa600640 1 -- 172.21.15.46:0/2728992528 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc09c8f0 con 0x56059f55fb50 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f51fa600640 1 -- 172.21.15.46:0/2728992528 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc0d8040 con 0x56059f55fb50 2024-04-17T08:00:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.380+0000 7f51fa600640 1 -- 172.21.15.46:0/2728992528 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc09c480 con 0x56059f55fb50 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f520e000640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f51f00a3bb0 0x7f51f00a6070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f520002c640 tx=0x7f5200032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 -- 172.21.15.46:0/2728992528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 msgr2=0x56059f564650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 --2- 172.21.15.46:0/2728992528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f564650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51fc09c820 tx=0x7f51fc0d95d0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 -- 172.21.15.46:0/2728992528 shutdown_connections 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 --2- 172.21.15.46:0/2728992528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f567150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 --2- 172.21.15.46:0/2728992528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f564650 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 --2- 172.21.15.46:0/2728992528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f55f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 -- 172.21.15.46:0/2728992528 >> 172.21.15.46:0/2728992528 conn(0x56059f55a6d0 msgr2=0x56059f55aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.381+0000 7f5212d82740 1 -- 172.21.15.46:0/2728992528 shutdown_connections 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.382+0000 7f5212d82740 1 -- 172.21.15.46:0/2728992528 wait complete. 2024-04-17T08:00:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.382+0000 7f5212d82740 1 -- start start 2024-04-17T08:00:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.382+0000 7f5212d82740 1 Processor -- start 2024-04-17T08:00:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f5212d82740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f5696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f5212d82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f569c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f5212d82740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f570c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f5212d82740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56059f567c70 con 0x56059f564cc0 2024-04-17T08:00:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f5212d82740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56059f47e830 con 0x56059f55f160 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f5212d82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56059f567df0 con 0x56059f55fb50 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f5212d82740 10 client.0 _ll_register_callbacks cb 0x56059f44cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f520d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f5696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f520d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f5696d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51904/0 (socket says 172.21.15.46:51904) 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f520d600640 1 -- 172.21.15.46:0/4028824477 learned_addr learned my addr 172.21.15.46:0/4028824477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f520cc00640 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f569c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.383+0000 7f520e000640 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f570c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f520d600640 1 -- 172.21.15.46:0/4028824477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 msgr2=0x56059f569c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f520d600640 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f569c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f520d600640 1 -- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 msgr2=0x56059f570c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f520d600640 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f570c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f520d600640 1 -- 172.21.15.46:0/4028824477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059f482920 con 0x56059f55f160 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f520d600640 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f5696d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0994f0 tx=0x7f51fc00cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b167dcbe6dfe0dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f51eea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f51eea00640 1 -- 172.21.15.46:0/4028824477 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc09c660 con 0x56059f55f160 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f51eea00640 1 -- 172.21.15.46:0/4028824477 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f51fc0d46b0 con 0x56059f55f160 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51fc0d8040 con 0x56059f55f160 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f51eea00640 1 -- 172.21.15.46:0/4028824477 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51fc09c480 con 0x56059f55f160 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f5212d82740 10 client.25955 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.384+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56059f4507f0 con 0x56059f55f160 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f51eea00640 1 -- 172.21.15.46:0/4028824477 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc0d8040 con 0x56059f55f160 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f51eea00640 1 -- 172.21.15.46:0/4028824477 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f51fc09cd80 con 0x56059f55f160 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f5212d82740 10 client.25955 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f5212d82740 10 client.25955 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f5212d82740 20 client.25955 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f5212d82740 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f574db0 0x56059f5771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f5212d82740 4 client.25955 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059f4d2a40 con 0x56059f574db0 2024-04-17T08:00:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.385+0000 7f520cc00640 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f574db0 0x56059f5771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.386+0000 7f520cc00640 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f574db0 0x56059f5771a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2632b9d6ffc367b server_cookie=b158eefbc602cc0 in_seq=0 out_seq=0 2024-04-17T08:00:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.386+0000 7f51eea00640 10 client.25955 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f51eea00640 1 -- 172.21.15.46:0/4028824477 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56059f4d2a40 con 0x56059f574db0 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f51eea00640 10 client.25955 handle_command_reply: tid=0 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 client.25955 shutdown 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 msgr2=0x56059f5696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f5696d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0994f0 tx=0x7f51fc00cf20 comp rx=0 tx=0).stop 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 shutdown_connections 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f574db0 0x56059f5771a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f570c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f569c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 --2- 172.21.15.46:0/4028824477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f5696d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f51eea00640 0 client.25955 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:31.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.387+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 >> 172.21.15.46:0/4028824477 conn(0x56059f55a6d0 msgr2=0x56059f559930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.388+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 shutdown_connections 2024-04-17T08:00:31.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.388+0000 7f5212d82740 1 -- 172.21.15.46:0/4028824477 wait complete. 2024-04-17T08:00:31.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.388+0000 7f5212d82740 20 client.25955 trim_cache size 0 max 16384 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 Processor -- start 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 -- start start 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f5699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f570be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f5590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52081b3930 con 0x56059f564cc0 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52081b11f0 con 0x56059f55fb50 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f5212d82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52081af040 con 0x56059f55f160 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f520cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f570be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f520e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f5590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f520cc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f570be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51906/0 (socket says 172.21.15.46:51906) 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.403+0000 7f520cc00640 1 -- 172.21.15.46:0/2122572725 learned_addr learned my addr 172.21.15.46:0/2122572725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f520d600640 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f5699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f520cc00640 1 -- 172.21.15.46:0/2122572725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 msgr2=0x56059f5699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f520cc00640 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f5699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f520cc00640 1 -- 172.21.15.46:0/2122572725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 msgr2=0x56059f5590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f520cc00640 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f5590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f520cc00640 1 -- 172.21.15.46:0/2122572725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059f1d73f0 con 0x56059f55fb50 2024-04-17T08:00:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f520cc00640 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f570be0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56059f5779f0 tx=0x7f51f4010b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d504505ec8c3b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f51f9c00640 1 -- 172.21.15.46:0/2122572725 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4024070 con 0x56059f55fb50 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f51f9c00640 1 -- 172.21.15.46:0/2122572725 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f400f8f0 con 0x56059f55fb50 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.404+0000 7f51f9c00640 1 -- 172.21.15.46:0/2122572725 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4003970 con 0x56059f55fb50 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 -- 172.21.15.46:0/2122572725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 msgr2=0x56059f570be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f570be0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56059f5779f0 tx=0x7f51f4010b90 comp rx=0 tx=0).stop 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 -- 172.21.15.46:0/2122572725 shutdown_connections 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f5590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55fb50 0x56059f570be0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 --2- 172.21.15.46:0/2122572725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55f160 0x56059f5699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 -- 172.21.15.46:0/2122572725 >> 172.21.15.46:0/2122572725 conn(0x56059f55a6d0 msgr2=0x56059f5692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 -- 172.21.15.46:0/2122572725 shutdown_connections 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.405+0000 7f5212d82740 1 -- 172.21.15.46:0/2122572725 wait complete. 2024-04-17T08:00:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 -- start start 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 Processor -- start 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f56df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f56e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f56f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52081af6b0 con 0x56059f564cc0 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52081b0b20 con 0x56059f55f160 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52081afd80 con 0x56059f55fb50 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.406+0000 7f5212d82740 10 client.0 _ll_register_callbacks cb 0x7f51fc0d4b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f56df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f56f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f56e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f56df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51920/0 (socket says 172.21.15.46:51920) 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f56e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55738/0 (socket says 172.21.15.46:55738) 2024-04-17T08:00:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 -- 172.21.15.46:0/3149880502 learned_addr learned my addr 172.21.15.46:0/3149880502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 -- 172.21.15.46:0/3149880502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 msgr2=0x56059f56e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f56e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 -- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 msgr2=0x56059f56f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f56f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 -- 172.21.15.46:0/3149880502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059f2a17d0 con 0x56059f55f160 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520e000640 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f56f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f520d600640 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f56df10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0a51b0 tx=0x7f51fc0a55a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a489515254512daa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.407+0000 7f51eea00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f51eea00640 1 -- 172.21.15.46:0/3149880502 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc09c8f0 con 0x56059f55f160 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f51eea00640 1 -- 172.21.15.46:0/3149880502 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f51fc09cd80 con 0x56059f55f160 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f400f8f0 con 0x56059f55f160 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f51eea00640 1 -- 172.21.15.46:0/3149880502 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51fc0d4450 con 0x56059f55f160 2024-04-17T08:00:31.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 10 client.25961 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56059f570b10 con 0x56059f55f160 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f51eea00640 1 -- 172.21.15.46:0/3149880502 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f400f8f0 con 0x56059f55f160 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f51eea00640 1 -- 172.21.15.46:0/3149880502 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f51fc0b1080 con 0x56059f55f160 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 10 client.25961 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 10 client.25961 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 20 client.25961 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f5db330 0x56059f5dd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 4 client.25961 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.408+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56059f4cc8b0 con 0x56059f5db330 2024-04-17T08:00:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.409+0000 7f520cc00640 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f5db330 0x56059f5dd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.409+0000 7f520cc00640 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f5db330 0x56059f5dd720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5add8f49a492f6f server_cookie=573f7c6b350659ab in_seq=0 out_seq=0 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.409+0000 7f51eea00640 10 client.25961 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f51eea00640 1 -- 172.21.15.46:0/3149880502 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56059f4cc8b0 con 0x56059f5db330 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f51eea00640 10 client.25961 handle_command_reply: tid=0 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 client.25961 shutdown 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 msgr2=0x56059f56df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f56df10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0a51b0 tx=0x7f51fc0a55a0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 shutdown_connections 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x56059f5db330 0x56059f5dd720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56059f564cc0 0x56059f56f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56059f55fb50 0x56059f56e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f51eea00640 0 client.25961 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:00:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 --2- 172.21.15.46:0/3149880502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56059f55f160 0x56059f56df10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 >> 172.21.15.46:0/3149880502 conn(0x56059f55a6d0 msgr2=0x56059f55d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 shutdown_connections 2024-04-17T08:00:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 1 -- 172.21.15.46:0/3149880502 wait complete. 2024-04-17T08:00:31.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.410+0000 7f5212d82740 20 client.25961 trim_cache size 0 max 16384 2024-04-17T08:00:31.415 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f51f00a3bb0 msgr2=0x7f51f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f51f00a3bb0 0x7f51f00a6070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f520002c640 tx=0x7f5200032590 comp rx=0 tx=0).stop 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208076920 msgr2=0x7f520819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208076920 0x7f520819e660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9f0 tx=0x7f51fc00eeb0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 shutdown_connections 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f51f00a3bb0 0x7f51f00a6070 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5208078500 0x7f52081a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5208077b60 0x7f520819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 --2- 172.21.15.46:0/3997593995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5208076920 0x7f520819e660 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 >> 172.21.15.46:0/3997593995 conn(0x7f52080ff570 msgr2=0x7f5208101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 shutdown_connections 2024-04-17T08:00:31.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.411+0000 7f520ea00640 1 -- 172.21.15.46:0/3997593995 wait complete. 2024-04-17T08:00:31.430 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.707+0000 7f4562000640 1 -- 172.21.15.46:0/271709989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c1051d0 msgr2=0x7f455c105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.707+0000 7f4562000640 1 --2- 172.21.15.46:0/271709989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c1051d0 0x7f455c105650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f454402bc30 comp rx=0 tx=0).stop 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.707+0000 7f4562000640 1 -- 172.21.15.46:0/271709989 shutdown_connections 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.707+0000 7f4562000640 1 --2- 172.21.15.46:0/271709989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f455c105b90 0x7f455c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.707+0000 7f4562000640 1 --2- 172.21.15.46:0/271709989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c1051d0 0x7f455c105650 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.707+0000 7f4562000640 1 --2- 172.21.15.46:0/271709989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 0x7f455c104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.707+0000 7f4562000640 1 -- 172.21.15.46:0/271709989 >> 172.21.15.46:0/271709989 conn(0x7f455c0ff570 msgr2=0x7f455c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.708+0000 7f4562000640 1 -- 172.21.15.46:0/271709989 shutdown_connections 2024-04-17T08:00:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.708+0000 7f4562000640 1 -- 172.21.15.46:0/271709989 wait complete. 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 Processor -- start 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 -- start start 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 0x7f455c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f455c1051d0 0x7f455c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c105b90 0x7f455c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10fcc0 con 0x7f455c105b90 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10fb40 con 0x7f455c103fe0 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4562000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10fe40 con 0x7f455c1051d0 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.709+0000 7f4561600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c105b90 0x7f455c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 0x7f455c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 0x7f455c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51940/0 (socket says 172.21.15.46:51940) 2024-04-17T08:00:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4561600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c105b90 0x7f455c1a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36904/0 (socket says 172.21.15.46:36904) 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 -- 172.21.15.46:0/8433868 learned_addr learned my addr 172.21.15.46:0/8433868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f455be00640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f455c1051d0 0x7f455c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 -- 172.21.15.46:0/8433868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f455c1051d0 msgr2=0x7f455c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f455c1051d0 0x7f455c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 -- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c105b90 msgr2=0x7f455c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c105b90 0x7f455c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 -- 172.21.15.46:0/8433868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c1a1f40 con 0x7f455c103fe0 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4561600640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c105b90 0x7f455c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.710+0000 7f4560c00640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 0x7f455c19a320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f454c009d20 tx=0x7f454c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.711+0000 7f4559600640 1 -- 172.21.15.46:0/8433868 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c010630 con 0x7f455c103fe0 2024-04-17T08:00:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.711+0000 7f4559600640 1 -- 172.21.15.46:0/8433868 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c010cb0 con 0x7f455c103fe0 2024-04-17T08:00:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.711+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1a2230 con 0x7f455c103fe0 2024-04-17T08:00:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.711+0000 7f4559600640 1 -- 172.21.15.46:0/8433868 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c018de0 con 0x7f455c103fe0 2024-04-17T08:00:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.711+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a27a0 con 0x7f455c103fe0 2024-04-17T08:00:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.715+0000 7f45663dc740 1 Processor -- start 2024-04-17T08:00:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.715+0000 7f45663dc740 1 -- start start 2024-04-17T08:00:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f45663dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f45663dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562c4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f45663dc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f45663dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559356225020 con 0x5593562bfb50 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f45663dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5593561cb460 con 0x5593562bf160 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f45663dc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559356220fe0 con 0x5593562c4cc0 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f4559600640 1 -- 172.21.15.46:0/8433868 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f454c01f020 con 0x7f455c103fe0 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f455be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f4561600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f455be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55758/0 (socket says 172.21.15.46:55758) 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f455be00640 1 -- 172.21.15.46:0/1785495115 learned_addr learned my addr 172.21.15.46:0/1785495115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f4560c00640 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f455be00640 1 -- 172.21.15.46:0/1785495115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 msgr2=0x5593562bf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f455be00640 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562bf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.716+0000 7f455be00640 1 -- 172.21.15.46:0/1785495115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 msgr2=0x5593562c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.717+0000 7f455be00640 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.717+0000 7f455be00640 1 -- 172.21.15.46:0/1785495115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593561d3040 con 0x5593562c4cc0 2024-04-17T08:00:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.717+0000 7f455be00640 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c7150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f45440095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6318317606867e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.717+0000 7f4551c00640 1 -- 172.21.15.46:0/1785495115 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544045070 con 0x5593562c4cc0 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.717+0000 7f4551c00640 1 -- 172.21.15.46:0/1785495115 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544004040 con 0x5593562c4cc0 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.717+0000 7f4551c00640 1 -- 172.21.15.46:0/1785495115 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544004eb0 con 0x5593562c4cc0 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f4559600640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45400a3c30 0x7f45400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 -- 172.21.15.46:0/1785495115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 msgr2=0x5593562c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f4559600640 1 -- 172.21.15.46:0/8433868 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f454c0d0be0 con 0x7f455c103fe0 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c7150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f45440095e0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f4561600640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45400a3c30 0x7f45400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 -- 172.21.15.46:0/1785495115 shutdown_connections 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c7150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 --2- 172.21.15.46:0/1785495115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562bf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 -- 172.21.15.46:0/1785495115 >> 172.21.15.46:0/1785495115 conn(0x5593562ba6d0 msgr2=0x5593562baad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 -- 172.21.15.46:0/1785495115 shutdown_connections 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.718+0000 7f45663dc740 1 -- 172.21.15.46:0/1785495115 wait complete. 2024-04-17T08:00:31.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.719+0000 7f4561600640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45400a3c30 0x7f45400a60f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5593562c03b0 tx=0x7f4554005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.719+0000 7f45663dc740 1 -- start start 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.719+0000 7f45663dc740 1 Processor -- start 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.719+0000 7f45663dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f45663dc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bfb50 0x5593562c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f45663dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f45663dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593562c7c10 con 0x5593562bf160 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f45663dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5593561de830 con 0x5593562c4cc0 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f45663dc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5593562c7d90 con 0x5593562bfb50 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f45663dc740 10 client.0 _ll_register_callbacks cb 0x5593561acfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f4561600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f4560c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f4561600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:51968/0 (socket says 172.21.15.46:51968) 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f4561600640 1 -- 172.21.15.46:0/243956237 learned_addr learned my addr 172.21.15.46:0/243956237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f455be00640 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bfb50 0x5593562c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f455be00640 1 -- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 msgr2=0x5593562d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f455be00640 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f455be00640 1 -- 172.21.15.46:0/243956237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bf160 msgr2=0x5593562c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f455be00640 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f455be00640 1 -- 172.21.15.46:0/243956237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593561e2920 con 0x5593562bfb50 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.720+0000 7f4561600640 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f455be00640 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bfb50 0x5593562c9b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f454402c8e0 tx=0x7f4544004e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e366dbb1de2b1133 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f454a000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:31.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:31 smithi053 ceph-mon[36437]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:00:31.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:31 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/87593383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:00:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f454a000640 1 -- 172.21.15.46:0/243956237 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454404e070 con 0x5593562bfb50 2024-04-17T08:00:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4544004040 con 0x5593562bfb50 2024-04-17T08:00:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f454a000640 1 -- 172.21.15.46:0/243956237 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4544004eb0 con 0x5593562bfb50 2024-04-17T08:00:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f454a000640 1 -- 172.21.15.46:0/243956237 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4544059050 con 0x5593562bfb50 2024-04-17T08:00:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f45663dc740 10 client.25954 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.721+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593561b07f0 con 0x5593562bfb50 2024-04-17T08:00:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f454a000640 1 -- 172.21.15.46:0/243956237 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544004040 con 0x5593562bfb50 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f454a000640 1 -- 172.21.15.46:0/243956237 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f4544003b10 con 0x5593562bfb50 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f45663dc740 10 client.25954 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f45663dc740 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f45663dc740 20 client.25954 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f45663dc740 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5593562d4ca0 0x5593562d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f45663dc740 4 client.25954 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559356232a40 con 0x5593562d4ca0 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.722+0000 7f4560c00640 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5593562d4ca0 0x5593562d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.723+0000 7f4560c00640 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5593562d4ca0 0x5593562d7090 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b462f3f4c773aa0b server_cookie=64d7d9d59adee7e6 in_seq=0 out_seq=0 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.723+0000 7f454a000640 10 client.25954 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f454a000640 1 -- 172.21.15.46:0/243956237 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559356232a40 con 0x5593562d4ca0 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f454a000640 10 client.25954 handle_command_reply: tid=0 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 client.25954 shutdown 2024-04-17T08:00:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bfb50 msgr2=0x5593562c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bfb50 0x5593562c9b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f454402c8e0 tx=0x7f4544004e20 comp rx=0 tx=0).stop 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 shutdown_connections 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5593562d4ca0 0x5593562d7090 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bfb50 0x5593562c9b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 --2- 172.21.15.46:0/243956237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f454a000640 0 client.25954 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.724+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 >> 172.21.15.46:0/243956237 conn(0x5593562ba6d0 msgr2=0x5593562b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.725+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 shutdown_connections 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.725+0000 7f45663dc740 1 -- 172.21.15.46:0/243956237 wait complete. 2024-04-17T08:00:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.725+0000 7f45663dc740 20 client.25954 trim_cache size 0 max 16384 2024-04-17T08:00:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.740+0000 7f45663dc740 1 Processor -- start 2024-04-17T08:00:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.740+0000 7f45663dc740 1 -- start start 2024-04-17T08:00:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.740+0000 7f45663dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.740+0000 7f45663dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f45663dc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f45663dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1af510 con 0x5593562bfb50 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f45663dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1acdd0 con 0x5593562bf160 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f45663dc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1aac20 con 0x5593562c4cc0 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36934/0 (socket says 172.21.15.46:36934) 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 -- 172.21.15.46:0/557110921 learned_addr learned my addr 172.21.15.46:0/557110921 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f4560c00640 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f4561600640 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 -- 172.21.15.46:0/557110921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 msgr2=0x5593562c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 -- 172.21.15.46:0/557110921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 msgr2=0x5593562c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.741+0000 7f455be00640 1 -- 172.21.15.46:0/557110921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559355f373f0 con 0x5593562bfb50 2024-04-17T08:00:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.742+0000 7f455be00640 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562d0b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4544003f60 tx=0x7f45440095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8a2a8429b482e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.742+0000 7f4551200640 1 -- 172.21.15.46:0/557110921 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454404e030 con 0x5593562bfb50 2024-04-17T08:00:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.742+0000 7f4551200640 1 -- 172.21.15.46:0/557110921 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544004040 con 0x5593562bfb50 2024-04-17T08:00:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.742+0000 7f4551200640 1 -- 172.21.15.46:0/557110921 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544045040 con 0x5593562bfb50 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 -- 172.21.15.46:0/557110921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 msgr2=0x5593562d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562d0b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4544003f60 tx=0x7f45440095e0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 -- 172.21.15.46:0/557110921 shutdown_connections 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562c4cc0 0x5593562c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562bfb50 0x5593562d0b00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 --2- 172.21.15.46:0/557110921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562bf160 0x5593562c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 -- 172.21.15.46:0/557110921 >> 172.21.15.46:0/557110921 conn(0x5593562ba6d0 msgr2=0x5593562c1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 -- 172.21.15.46:0/557110921 shutdown_connections 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 -- 172.21.15.46:0/557110921 wait complete. 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 -- start start 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.743+0000 7f45663dc740 1 Processor -- start 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f45663dc740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bf160 0x5593562ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f45663dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f45663dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562cf6c0 0x5593562cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f45663dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ab290 con 0x5593562cf6c0 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f45663dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ac700 con 0x5593562c4cc0 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f45663dc740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1ab960 con 0x5593562bf160 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f45663dc740 10 client.0 _ll_register_callbacks cb 0x7f45440046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f4560c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bf160 0x5593562ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f4561600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562cf6c0 0x5593562cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f455be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f4560c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bf160 0x5593562ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55776/0 (socket says 172.21.15.46:55776) 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f4560c00640 1 -- 172.21.15.46:0/392023347 learned_addr learned my addr 172.21.15.46:0/392023347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f4560c00640 1 -- 172.21.15.46:0/392023347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 msgr2=0x5593562ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.744+0000 7f4560c00640 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f4560c00640 1 -- 172.21.15.46:0/392023347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562cf6c0 msgr2=0x5593562cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f4560c00640 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562cf6c0 0x5593562cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f4560c00640 1 -- 172.21.15.46:0/392023347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593560017d0 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f4561600640 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562cf6c0 0x5593562cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f4560c00640 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bf160 0x5593562ce1c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f454c09ca60 tx=0x7f454c018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5694c38a2711c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f454a000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f454a000640 1 -- 172.21.15.46:0/392023347 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c010e60 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f454a000640 1 -- 172.21.15.46:0/392023347 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f454c0bd0b0 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4544004040 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f454a000640 1 -- 172.21.15.46:0/392023347 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f454c09cad0 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f45663dc740 10 client.25960 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.745+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55935633b340 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f454a000640 1 -- 172.21.15.46:0/392023347 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544004040 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f454a000640 1 -- 172.21.15.46:0/392023347 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f454c099140 con 0x5593562bf160 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f45663dc740 10 client.25960 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f45663dc740 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f45663dc740 20 client.25960 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f45663dc740 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55935633bca0 0x55935633e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f45663dc740 4 client.25960 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:00:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55935622c8b0 con 0x55935633bca0 2024-04-17T08:00:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.746+0000 7f455be00640 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55935633bca0 0x55935633e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f455be00640 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55935633bca0 0x55935633e0c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6797df22aabbed server_cookie=4a67de9d28f2034d in_seq=0 out_seq=0 2024-04-17T08:00:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f454a000640 10 client.25960 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f454a000640 1 -- 172.21.15.46:0/392023347 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55935622c8b0 con 0x55935633bca0 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f454a000640 10 client.25960 handle_command_reply: tid=0 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 client.25960 shutdown 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bf160 msgr2=0x5593562ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bf160 0x5593562ce1c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f454c09ca60 tx=0x7f454c018420 comp rx=0 tx=0).stop 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 shutdown_connections 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55935633bca0 0x55935633e0c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593562cf6c0 0x5593562cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593562c4cc0 0x5593562ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 --2- 172.21.15.46:0/392023347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5593562bf160 0x5593562ce1c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 >> 172.21.15.46:0/392023347 conn(0x5593562ba6d0 msgr2=0x5593562c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f454a000640 0 client.25960 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.747+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 shutdown_connections 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f45663dc740 1 -- 172.21.15.46:0/392023347 wait complete. 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f45663dc740 20 client.25960 trim_cache size 0 max 16384 2024-04-17T08:00:31.752 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45400a3c30 msgr2=0x7f45400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45400a3c30 0x7f45400a60f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5593562c03b0 tx=0x7f4554005cc0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 msgr2=0x7f455c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 0x7f455c19a320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f454c009d20 tx=0x7f454c00c6a0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 shutdown_connections 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45400a3c30 0x7f45400a60f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f455c105b90 0x7f455c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f455c1051d0 0x7f455c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 --2- 172.21.15.46:0/8433868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f455c103fe0 0x7f455c19a320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.748+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 >> 172.21.15.46:0/8433868 conn(0x7f455c0ff570 msgr2=0x7f455c108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.749+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 shutdown_connections 2024-04-17T08:00:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:31.749+0000 7f4562000640 1 -- 172.21.15.46:0/8433868 wait complete. 2024-04-17T08:00:31.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:31 smithi062 ceph-mon[36507]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:00:31.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:31 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/87593383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:00:32.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:32 smithi046 ceph-mon[32038]: pgmap v478: 129 pgs: 129 active+clean; 14 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-17T08:00:32.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:32 smithi053 ceph-mon[36437]: pgmap v478: 129 pgs: 129 active+clean; 14 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-17T08:00:32.769 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T08:00:32.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:00:32.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T08:00:32.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:32 smithi062 ceph-mon[36507]: pgmap v478: 129 pgs: 129 active+clean; 14 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.010+0000 7f1798200640 1 -- 172.21.15.46:0/4084968119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 msgr2=0x7f179010c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.010+0000 7f1798200640 1 --2- 172.21.15.46:0/4084968119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 0x7f179010c600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1788009900 tx=0x7f178802bc30 comp rx=0 tx=0).stop 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.011+0000 7f1798200640 1 -- 172.21.15.46:0/4084968119 shutdown_connections 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.011+0000 7f1798200640 1 --2- 172.21.15.46:0/4084968119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 0x7f179010c600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.011+0000 7f1798200640 1 --2- 172.21.15.46:0/4084968119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17901053b0 0x7f1790105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.011+0000 7f1798200640 1 --2- 172.21.15.46:0/4084968119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 0x7f17901045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.011+0000 7f1798200640 1 -- 172.21.15.46:0/4084968119 >> 172.21.15.46:0/4084968119 conn(0x7f17900ff750 msgr2=0x7f1790101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.011+0000 7f1798200640 1 -- 172.21.15.46:0/4084968119 shutdown_connections 2024-04-17T08:00:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.011+0000 7f1798200640 1 -- 172.21.15.46:0/4084968119 wait complete. 2024-04-17T08:00:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 Processor -- start 2024-04-17T08:00:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 -- start start 2024-04-17T08:00:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 0x7f179019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17901053b0 0x7f179019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 0x7f17901a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f179010fea0 con 0x7f17901053b0 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f179010fd20 con 0x7f17901041c0 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.012+0000 7f1798200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1790110020 con 0x7f1790105d70 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 0x7f179019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1794e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17901053b0 0x7f179019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1796200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 0x7f17901a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 0x7f179019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52012/0 (socket says 172.21.15.46:52012) 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 -- 172.21.15.46:0/3058438876 learned_addr learned my addr 172.21.15.46:0/3058438876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 -- 172.21.15.46:0/3058438876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 msgr2=0x7f17901a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 0x7f17901a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 -- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17901053b0 msgr2=0x7f179019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17901053b0 0x7f179019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1795800640 1 -- 172.21.15.46:0/3058438876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a2050 con 0x7f17901041c0 2024-04-17T08:00:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.013+0000 7f1796200640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 0x7f17901a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.014+0000 7f1795800640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 0x7f179019a430 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f178400eb70 tx=0x7f178400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.014+0000 7f178e000640 1 -- 172.21.15.46:0/3058438876 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784003b70 con 0x7f17901041c0 2024-04-17T08:00:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.014+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a2340 con 0x7f17901041c0 2024-04-17T08:00:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.014+0000 7f178e000640 1 -- 172.21.15.46:0/3058438876 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178400ee70 con 0x7f17901041c0 2024-04-17T08:00:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.014+0000 7f178e000640 1 -- 172.21.15.46:0/3058438876 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784010830 con 0x7f17901041c0 2024-04-17T08:00:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.014+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a2840 con 0x7f17901041c0 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.017+0000 7f179af8e740 1 Processor -- start 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.017+0000 7f179af8e740 1 -- start start 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.017+0000 7f178e000640 1 -- 172.21.15.46:0/3058438876 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1784004020 con 0x7f17901041c0 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f179af8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a2727ec0 0x5569a27282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f179af8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a272d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f179af8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a272da20 0x5569a272feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f179af8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569a268dcb0 con 0x5569a272da20 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f179af8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5569a2634370 con 0x5569a27288b0 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f179af8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5569a2689c70 con 0x5569a2727ec0 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a2727ec0 0x5569a27282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1794e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a272da20 0x5569a272feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1795800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a272d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a2727ec0 0x5569a27282e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55806/0 (socket says 172.21.15.46:55806) 2024-04-17T08:00:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 -- 172.21.15.46:0/2293933840 learned_addr learned my addr 172.21.15.46:0/2293933840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 -- 172.21.15.46:0/2293933840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 msgr2=0x5569a272d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 --2- 172.21.15.46:0/2293933840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a272d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 -- 172.21.15.46:0/2293933840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a272da20 msgr2=0x5569a272feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 --2- 172.21.15.46:0/2293933840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a272da20 0x5569a272feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.018+0000 7f1796200640 1 -- 172.21.15.46:0/2293933840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a2653910 con 0x5569a2727ec0 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.019+0000 7f1796200640 1 --2- 172.21.15.46:0/2293933840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a2727ec0 0x5569a27282e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f178802c240 tx=0x7f1788009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa96124a3ac13c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.019+0000 7f1782a00640 1 -- 172.21.15.46:0/2293933840 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788045070 con 0x5569a2727ec0 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.019+0000 7f1782a00640 1 -- 172.21.15.46:0/2293933840 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178802cd30 con 0x5569a2727ec0 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.019+0000 7f1782a00640 1 -- 172.21.15.46:0/2293933840 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788005150 con 0x5569a2727ec0 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.019+0000 7f178e000640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17780a3bb0 0x7f17780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.019+0000 7f178e000640 1 -- 172.21.15.46:0/3058438876 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f17840d0e10 con 0x7f17901041c0 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f1795800640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17780a3bb0 0x7f17780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 -- 172.21.15.46:0/2293933840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a2727ec0 msgr2=0x5569a27282e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 --2- 172.21.15.46:0/2293933840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a2727ec0 0x5569a27282e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f178802c240 tx=0x7f1788009610 comp rx=0 tx=0).stop 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 -- 172.21.15.46:0/2293933840 shutdown_connections 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 --2- 172.21.15.46:0/2293933840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a272da20 0x5569a272feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 --2- 172.21.15.46:0/2293933840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a272d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 --2- 172.21.15.46:0/2293933840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a2727ec0 0x5569a27282e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 -- 172.21.15.46:0/2293933840 >> 172.21.15.46:0/2293933840 conn(0x5569a2723430 msgr2=0x5569a2723830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 -- 172.21.15.46:0/2293933840 shutdown_connections 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f179af8e740 1 -- 172.21.15.46:0/2293933840 wait complete. 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.020+0000 7f1795800640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17780a3bb0 0x7f17780a6070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f17840ce8f0 tx=0x7f17840d48f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.021+0000 7f179af8e740 1 -- start start 2024-04-17T08:00:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.021+0000 7f179af8e740 1 Processor -- start 2024-04-17T08:00:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.021+0000 7f179af8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a27323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f179af8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2732920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f179af8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2739960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f179af8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569a2730970 con 0x5569a2727ec0 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f179af8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5569a2647730 con 0x5569a27288b0 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f179af8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5569a2730af0 con 0x5569a272da20 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f179af8e740 10 client.0 _ll_register_callbacks cb 0x5569a2649b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a27323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a27323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36972/0 (socket says 172.21.15.46:36972) 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 -- 172.21.15.46:0/4151536798 learned_addr learned my addr 172.21.15.46:0/4151536798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1794e00640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2732920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2739960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 -- 172.21.15.46:0/4151536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 msgr2=0x5569a2739960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2739960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 -- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 msgr2=0x5569a2732920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2732920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1796200640 1 -- 172.21.15.46:0/4151536798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a24687e0 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.022+0000 7f1794e00640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2732920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f1796200640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a27323e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1788036040 tx=0x7f1788005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7594add691f0c45b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f1777400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f1777400640 1 -- 172.21.15.46:0/4151536798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788050070 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f1777400640 1 -- 172.21.15.46:0/4151536798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f178804a650 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178802cd30 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f1777400640 1 -- 172.21.15.46:0/4151536798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f178805c050 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f179af8e740 10 client.15840 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.023+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569a24c8f70 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f1777400640 1 -- 172.21.15.46:0/4151536798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178802cd30 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f1777400640 1 -- 172.21.15.46:0/4151536798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f178803f7e0 con 0x5569a2727ec0 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f179af8e740 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f179af8e740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:33.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f179af8e740 20 client.15840 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f179af8e740 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a273de50 0x5569a2740240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f179af8e740 4 client.15840 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569a269b6d0 con 0x5569a273de50 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.024+0000 7f1794e00640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a273de50 0x5569a2740240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.025+0000 7f1794e00640 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a273de50 0x5569a2740240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20448c74e7b0db92 server_cookie=d78034b1e609befe in_seq=0 out_seq=0 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.025+0000 7f1777400640 10 client.15840 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f1777400640 1 -- 172.21.15.46:0/4151536798 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569a269b6d0 con 0x5569a273de50 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f1777400640 10 client.15840 handle_command_reply: tid=0 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 client.15840 shutdown 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 msgr2=0x5569a27323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a27323e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1788036040 tx=0x7f1788005e50 comp rx=0 tx=0).stop 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 shutdown_connections 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a273de50 0x5569a2740240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2739960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2732920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 --2- 172.21.15.46:0/4151536798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a27323e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f1777400640 0 client.15840 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 >> 172.21.15.46:0/4151536798 conn(0x5569a2723430 msgr2=0x5569a27225e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 shutdown_connections 2024-04-17T08:00:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 1 -- 172.21.15.46:0/4151536798 wait complete. 2024-04-17T08:00:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.026+0000 7f179af8e740 20 client.15840 trim_cache size 0 max 16384 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 Processor -- start 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 -- start start 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2732700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a27398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2721d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17901af5f0 con 0x5569a2727ec0 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17901aceb0 con 0x5569a27288b0 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.046+0000 7f179af8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17901aad00 con 0x5569a272da20 2024-04-17T08:00:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2732700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2732700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36974/0 (socket says 172.21.15.46:36974) 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 -- 172.21.15.46:0/3892279750 learned_addr learned my addr 172.21.15.46:0/3892279750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1794e00640 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a27398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2721d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 -- 172.21.15.46:0/3892279750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 msgr2=0x5569a2721d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2721d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 -- 172.21.15.46:0/3892279750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 msgr2=0x5569a27398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a27398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 -- 172.21.15.46:0/3892279750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a2360170 con 0x5569a2727ec0 2024-04-17T08:00:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.047+0000 7f1796200640 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2732700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f178802bc70 tx=0x7f1788005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1076d79d92e8a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f1782000640 1 -- 172.21.15.46:0/3892279750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788050070 con 0x5569a2727ec0 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f1782000640 1 -- 172.21.15.46:0/3892279750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178802cd30 con 0x5569a2727ec0 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f1782000640 1 -- 172.21.15.46:0/3892279750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788045040 con 0x5569a2727ec0 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 -- 172.21.15.46:0/3892279750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 msgr2=0x5569a2732700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2732700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f178802bc70 tx=0x7f1788005970 comp rx=0 tx=0).stop 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 -- 172.21.15.46:0/3892279750 shutdown_connections 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a2721d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a27398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 --2- 172.21.15.46:0/3892279750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2732700 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 -- 172.21.15.46:0/3892279750 >> 172.21.15.46:0/3892279750 conn(0x5569a2723430 msgr2=0x5569a2731ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.048+0000 7f179af8e740 1 -- 172.21.15.46:0/3892279750 shutdown_connections 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.049+0000 7f179af8e740 1 -- 172.21.15.46:0/3892279750 wait complete. 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.049+0000 7f179af8e740 1 -- start start 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.049+0000 7f179af8e740 1 Processor -- start 2024-04-17T08:00:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.049+0000 7f179af8e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2737310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.049+0000 7f179af8e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2737850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f179af8e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a27387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f179af8e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17901ab370 con 0x5569a2727ec0 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f179af8e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17901ac7e0 con 0x5569a27288b0 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f179af8e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17901aba40 con 0x5569a272da20 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f179af8e740 10 client.0 _ll_register_callbacks cb 0x7f177c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2737310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2737310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36978/0 (socket says 172.21.15.46:36978) 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 -- 172.21.15.46:0/1180111186 learned_addr learned my addr 172.21.15.46:0/1180111186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1794e00640 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2737850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 -- 172.21.15.46:0/1180111186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 msgr2=0x5569a27387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a27387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 -- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 msgr2=0x5569a2737850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2737850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 -- 172.21.15.46:0/1180111186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a2738db0 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1796200640 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2737310 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f178802c7b0 tx=0x7f1788004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd96c169f28ff64d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.050+0000 7f1777400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f1777400640 1 -- 172.21.15.46:0/1180111186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788045040 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f1777400640 1 -- 172.21.15.46:0/1180111186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f1788034600 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178802cd30 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f1777400640 1 -- 172.21.15.46:0/1180111186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1788050040 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f179af8e740 10 client.15846 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569a27a4490 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f1777400640 1 -- 172.21.15.46:0/1180111186 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178802cd30 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f1777400640 1 -- 172.21.15.46:0/1180111186 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f178804a450 con 0x5569a2727ec0 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.051+0000 7f179af8e740 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f179af8e740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f179af8e740 20 client.15846 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f179af8e740 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a27a4da0 0x5569a27a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f179af8e740 4 client.15846 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T08:00:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5569a2695540 con 0x5569a27a4da0 2024-04-17T08:00:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f1794e00640 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a27a4da0 0x5569a27a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f1794e00640 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a27a4da0 0x5569a27a71c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2e31a754c58b06 server_cookie=be06f6747e2785 in_seq=0 out_seq=0 2024-04-17T08:00:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.052+0000 7f1777400640 10 client.15846 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.053+0000 7f1777400640 1 -- 172.21.15.46:0/1180111186 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5569a2695540 con 0x5569a27a4da0 2024-04-17T08:00:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.053+0000 7f1777400640 10 client.15846 handle_command_reply: tid=0 2024-04-17T08:00:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 client.15846 shutdown 2024-04-17T08:00:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 msgr2=0x5569a2737310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2737310 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f178802c7b0 tx=0x7f1788004fe0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 shutdown_connections 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5569a27a4da0 0x5569a27a71c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5569a272da20 0x5569a27387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5569a27288b0 0x5569a2737850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f1777400640 0 client.15846 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 --2- 172.21.15.46:0/1180111186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a2727ec0 0x5569a2737310 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 >> 172.21.15.46:0/1180111186 conn(0x5569a2723430 msgr2=0x5569a2721d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 shutdown_connections 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 1 -- 172.21.15.46:0/1180111186 wait complete. 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.054+0000 7f179af8e740 20 client.15846 trim_cache size 0 max 16384 2024-04-17T08:00:33.057 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:00:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-17T08:00:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "7e00b3d8-0760-4141-8373-52eadfa3def0", 2024-04-17T08:00:33.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-17T08:00:33.058 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17780a3bb0 msgr2=0x7f17780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17780a3bb0 0x7f17780a6070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f17840ce8f0 tx=0x7f17840d48f0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 msgr2=0x7f179019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 0x7f179019a430 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f178400eb70 tx=0x7f178400c6a0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 shutdown_connections 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f17780a3bb0 0x7f17780a6070 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1790105d70 0x7f17901a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17901053b0 0x7f179019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 --2- 172.21.15.46:0/3058438876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17901041c0 0x7f179019a430 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 >> 172.21.15.46:0/3058438876 conn(0x7f17900ff750 msgr2=0x7f1790108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 shutdown_connections 2024-04-17T08:00:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.055+0000 7f1798200640 1 -- 172.21.15.46:0/3058438876 wait complete. 2024-04-17T08:00:33.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:00:33.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:00:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 -- 172.21.15.46:0/3980729708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 msgr2=0x7f83bc110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 --2- 172.21.15.46:0/3980729708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 0x7f83bc110820 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009ca0 tx=0x7f83b402c100 comp rx=0 tx=0).stop 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 -- 172.21.15.46:0/3980729708 shutdown_connections 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 --2- 172.21.15.46:0/3980729708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 0x7f83bc110820 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 --2- 172.21.15.46:0/3980729708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83bc1095d0 0x7f83bc109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 --2- 172.21.15.46:0/3980729708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1083e0 0x7f83bc108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 -- 172.21.15.46:0/3980729708 >> 172.21.15.46:0/3980729708 conn(0x7f83bc0773a0 msgr2=0x7f83bc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 -- 172.21.15.46:0/3980729708 shutdown_connections 2024-04-17T08:00:33.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.302+0000 7f83c1000640 1 -- 172.21.15.46:0/3980729708 wait complete. 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.303+0000 7f83c1000640 1 Processor -- start 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83c1000640 1 -- start start 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83c1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83bc1083e0 0x7f83bc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1095d0 0x7f83bc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83c1000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 0x7f83bc1a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83c1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc114070 con 0x7f83bc1095d0 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83c1000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc113ef0 con 0x7f83bc109f90 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83c1000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1141f0 con 0x7f83bc1083e0 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1095d0 0x7f83bc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1095d0 0x7f83bc19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36998/0 (socket says 172.21.15.46:36998) 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83baa00640 1 -- 172.21.15.46:0/3306776840 learned_addr learned my addr 172.21.15.46:0/3306776840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83bb400640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83bc1083e0 0x7f83bc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.304+0000 7f83bbe00640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 0x7f83bc1a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.305+0000 7f83baa00640 1 -- 172.21.15.46:0/3306776840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83bc1083e0 msgr2=0x7f83bc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.305+0000 7f83baa00640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83bc1083e0 0x7f83bc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.305+0000 7f83baa00640 1 -- 172.21.15.46:0/3306776840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 msgr2=0x7f83bc1a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.305+0000 7f83baa00640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 0x7f83bc1a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.305+0000 7f83baa00640 1 -- 172.21.15.46:0/3306776840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc1a62b0 con 0x7f83bc1095d0 2024-04-17T08:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.305+0000 7f83baa00640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1095d0 0x7f83bc19ebd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9f0 tx=0x7f83a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.305+0000 7f83b3e00640 1 -- 172.21.15.46:0/3306776840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a400cd80 con 0x7f83bc1095d0 2024-04-17T08:00:33.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.306+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc1a65a0 con 0x7f83bc1095d0 2024-04-17T08:00:33.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.306+0000 7f83b3e00640 1 -- 172.21.15.46:0/3306776840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a4005500 con 0x7f83bc1095d0 2024-04-17T08:00:33.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.306+0000 7f83b3e00640 1 -- 172.21.15.46:0/3306776840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4010690 con 0x7f83bc1095d0 2024-04-17T08:00:33.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.306+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc1a6af0 con 0x7f83bc1095d0 2024-04-17T08:00:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 Processor -- start 2024-04-17T08:00:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 -- start start 2024-04-17T08:00:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83b3e00640 1 -- 172.21.15.46:0/3306776840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f83a40046c0 con 0x7f83bc1095d0 2024-04-17T08:00:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79df370 0x5600b79df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79dfd60 0x5600b79e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79e7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600b7946120 con 0x5600b79e4ed0 2024-04-17T08:00:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5600b78ec340 con 0x5600b79dfd60 2024-04-17T08:00:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.309+0000 7f83c524c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5600b79420e0 con 0x5600b79df370 2024-04-17T08:00:33.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79dfd60 0x5600b79e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79dfd60 0x5600b79e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:52080/0 (socket says 172.21.15.46:52080) 2024-04-17T08:00:33.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 -- 172.21.15.46:0/492268029 learned_addr learned my addr 172.21.15.46:0/492268029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83baa00640 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bbe00640 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79df370 0x5600b79df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 -- 172.21.15.46:0/492268029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79df370 msgr2=0x5600b79df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79df370 0x5600b79df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 -- 172.21.15.46:0/492268029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 msgr2=0x5600b79e7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79e7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.310+0000 7f83bb400640 1 -- 172.21.15.46:0/492268029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b77227d0 con 0x5600b79dfd60 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.311+0000 7f83bb400640 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79dfd60 0x5600b79e4860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009ca0 tx=0x7f83ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7568c1068b13773 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.311+0000 7f83b0c00640 1 -- 172.21.15.46:0/492268029 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac004220 con 0x5600b79dfd60 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.311+0000 7f83b0c00640 1 -- 172.21.15.46:0/492268029 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ac0043c0 con 0x5600b79dfd60 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.311+0000 7f83b0c00640 1 -- 172.21.15.46:0/492268029 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac019930 con 0x5600b79dfd60 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.311+0000 7f83b3e00640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83a00a3bb0 0x7f83a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.311+0000 7f83b3e00640 1 -- 172.21.15.46:0/3306776840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f83a40d0bf0 con 0x7f83bc1095d0 2024-04-17T08:00:33.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.311+0000 7f83bbe00640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83a00a3bb0 0x7f83a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 -- 172.21.15.46:0/492268029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79dfd60 msgr2=0x5600b79e4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79dfd60 0x5600b79e4860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009ca0 tx=0x7f83ac011810 comp rx=0 tx=0).stop 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 -- 172.21.15.46:0/492268029 shutdown_connections 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79dfd60 0x5600b79e4860 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 --2- 172.21.15.46:0/492268029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79df370 0x5600b79df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 -- 172.21.15.46:0/492268029 >> 172.21.15.46:0/492268029 conn(0x5600b79da8e0 msgr2=0x5600b79dace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 -- 172.21.15.46:0/492268029 shutdown_connections 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83bbe00640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83a00a3bb0 0x7f83a00a6070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f83b4005e30 tx=0x7f83b4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.312+0000 7f83c524c740 1 -- 172.21.15.46:0/492268029 wait complete. 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 -- start start 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 Processor -- start 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600b79e7e80 con 0x5600b79e4ed0 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5600b78ff710 con 0x5600b79df370 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.313+0000 7f83c524c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5600b79e8000 con 0x5600b79dfd60 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83c524c740 10 client.0 _ll_register_callbacks cb 0x5600b78faaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79e9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:55870/0 (socket says 172.21.15.46:55870) 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 -- 172.21.15.46:0/1178949688 learned_addr learned my addr 172.21.15.46:0/1178949688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83bbe00640 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 -- 172.21.15.46:0/1178949688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 msgr2=0x5600b79e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 -- 172.21.15.46:0/1178949688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 msgr2=0x5600b79f0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 -- 172.21.15.46:0/1178949688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b76583f0 con 0x5600b79dfd60 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83baa00640 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79e9de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83a4098f40 tx=0x7f83a4098e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f8e7de589b5103 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.314+0000 7f83a9600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83a9600640 1 -- 172.21.15.46:0/1178949688 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a409d1e0 con 0x5600b79dfd60 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83a9600640 1 -- 172.21.15.46:0/1178949688 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f83a40ce410 con 0x5600b79dfd60 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ac0043c0 con 0x5600b79dfd60 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83a9600640 1 -- 172.21.15.46:0/1178949688 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83a409cae0 con 0x5600b79dfd60 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83c524c740 10 client.25987 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600b76180f0 con 0x5600b79dfd60 2024-04-17T08:00:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83a9600640 1 -- 172.21.15.46:0/1178949688 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ac0043c0 con 0x5600b79dfd60 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83a9600640 1 -- 172.21.15.46:0/1178949688 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f83a409bf30 con 0x5600b79dfd60 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83c524c740 10 client.25987 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.315+0000 7f83c524c740 10 client.25987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.316+0000 7f83c524c740 20 client.25987 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.316+0000 7f83c524c740 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b79f5690 0x5600b79f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.316+0000 7f83c524c740 4 client.25987 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.316+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600b7953b40 con 0x5600b79f5690 2024-04-17T08:00:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.316+0000 7f83bb400640 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b79f5690 0x5600b79f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.317+0000 7f83bb400640 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b79f5690 0x5600b79f7b50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e53a7e57350eef server_cookie=470d7c2c7071f9d8 in_seq=0 out_seq=0 2024-04-17T08:00:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.317+0000 7f83a9600640 10 client.25987 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.317+0000 7f83a9600640 1 -- 172.21.15.46:0/1178949688 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600b7953b40 con 0x5600b79f5690 2024-04-17T08:00:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.317+0000 7f83a9600640 10 client.25987 handle_command_reply: tid=0 2024-04-17T08:00:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.317+0000 7f83c524c740 1 client.25987 shutdown 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 msgr2=0x5600b79e9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79e9de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83a4098f40 tx=0x7f83a4098e40 comp rx=0 tx=0).stop 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 shutdown_connections 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b79f5690 0x5600b79f7b50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79e9de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 --2- 172.21.15.46:0/1178949688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 >> 172.21.15.46:0/1178949688 conn(0x5600b79da8e0 msgr2=0x5600b79d9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83a9600640 0 client.25987 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 shutdown_connections 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 1 -- 172.21.15.46:0/1178949688 wait complete. 2024-04-17T08:00:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.318+0000 7f83c524c740 20 client.25987 trim_cache size 0 max 16384 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.337+0000 7f83c524c740 1 Processor -- start 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.337+0000 7f83c524c740 1 -- start start 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.337+0000 7f83c524c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79dacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83c524c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83c524c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83c524c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b3860 con 0x5600b79e4ed0 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83c524c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b1120 con 0x5600b79df370 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83c524c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1aef70 con 0x5600b79dfd60 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79de250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37022/0 (socket says 172.21.15.46:37022) 2024-04-17T08:00:33.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 -- 172.21.15.46:0/2881414648 learned_addr learned my addr 172.21.15.46:0/2881414648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bb400640 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79dacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83baa00640 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 -- 172.21.15.46:0/2881414648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 msgr2=0x5600b79f0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79f0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 -- 172.21.15.46:0/2881414648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 msgr2=0x5600b79dacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79dacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bbe00640 1 -- 172.21.15.46:0/2881414648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b79db650 con 0x5600b79e4ed0 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.338+0000 7f83bb400640 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79dacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.339+0000 7f83bbe00640 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79de250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009ca0 tx=0x7f83b4041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257c68257f41f517 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.339+0000 7f83abe00640 1 -- 172.21.15.46:0/2881414648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4039040 con 0x5600b79e4ed0 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.339+0000 7f83abe00640 1 -- 172.21.15.46:0/2881414648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b4004040 con 0x5600b79e4ed0 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.339+0000 7f83abe00640 1 -- 172.21.15.46:0/2881414648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b404e6a0 con 0x5600b79e4ed0 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 -- 172.21.15.46:0/2881414648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 msgr2=0x5600b79de250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79de250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009ca0 tx=0x7f83b4041000 comp rx=0 tx=0).stop 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 -- 172.21.15.46:0/2881414648 shutdown_connections 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79de250 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79f0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 --2- 172.21.15.46:0/2881414648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79dacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 -- 172.21.15.46:0/2881414648 >> 172.21.15.46:0/2881414648 conn(0x5600b79da8e0 msgr2=0x5600b79ddb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 -- 172.21.15.46:0/2881414648 shutdown_connections 2024-04-17T08:00:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.340+0000 7f83c524c740 1 -- 172.21.15.46:0/2881414648 wait complete. 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 -- start start 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 Processor -- start 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79ef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1af5e0 con 0x5600b79e4ed0 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b0a50 con 0x5600b79df370 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1afcb0 con 0x5600b79dfd60 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.341+0000 7f83c524c740 10 client.0 _ll_register_callbacks cb 0x7f83a40d4320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bb400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37028/0 (socket says 172.21.15.46:37028) 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 -- 172.21.15.46:0/1693040743 learned_addr learned my addr 172.21.15.46:0/1693040743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83baa00640 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 -- 172.21.15.46:0/1693040743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 msgr2=0x5600b79ef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79ef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 -- 172.21.15.46:0/1693040743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 msgr2=0x5600b79eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 -- 172.21.15.46:0/1693040743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b79f0740 con 0x5600b79e4ed0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83bbe00640 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f83b40049d0 tx=0x7f83b40048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fbda2a0771da79 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83a9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.342+0000 7f83a9600640 1 -- 172.21.15.46:0/1693040743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4039040 con 0x5600b79e4ed0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83a9600640 1 -- 172.21.15.46:0/1693040743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f83b4003a30 con 0x5600b79e4ed0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b4004040 con 0x5600b79e4ed0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83a9600640 1 -- 172.21.15.46:0/1693040743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83b4068050 con 0x5600b79e4ed0 2024-04-17T08:00:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83c524c740 10 client.15864 fetch_fsmap learned FSMap version 25 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600b7a5c3c0 con 0x5600b79e4ed0 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83a9600640 1 -- 172.21.15.46:0/1693040743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b4004040 con 0x5600b79e4ed0 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83a9600640 1 -- 172.21.15.46:0/1693040743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8351+0+0 (secure 0 0 0) 0x7f83b404e460 con 0x5600b79e4ed0 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83c524c740 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.343+0000 7f83c524c740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.344+0000 7f83c524c740 20 client.15864 populate_metadata read hostname 'smithi046' 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.344+0000 7f83c524c740 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b7a5ce20 0x5600b7a5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.344+0000 7f83c524c740 4 client.15864 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.344+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600b794d9b0 con 0x5600b7a5ce20 2024-04-17T08:00:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.344+0000 7f83bb400640 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b7a5ce20 0x5600b7a5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:00:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.344+0000 7f83bb400640 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b7a5ce20 0x5600b7a5f240 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369c3d293c76a5d0 server_cookie=f4cb13c02dd30461 in_seq=0 out_seq=0 2024-04-17T08:00:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.344+0000 7f83a9600640 10 client.15864 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83a9600640 1 -- 172.21.15.46:0/1693040743 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5600b794d9b0 con 0x5600b7a5ce20 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83a9600640 10 client.15864 handle_command_reply: tid=0 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 client.15864 shutdown 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 msgr2=0x5600b79f0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f83b40049d0 tx=0x7f83b40048b0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 shutdown_connections 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5600b7a5ce20 0x5600b7a5f240 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600b79e4ed0 0x5600b79f0150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83a9600640 0 client.15864 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b79dfd60 0x5600b79ef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 --2- 172.21.15.46:0/1693040743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5600b79df370 0x5600b79eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 >> 172.21.15.46:0/1693040743 conn(0x5600b79da8e0 msgr2=0x5600b79de3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 shutdown_connections 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 1 -- 172.21.15.46:0/1693040743 wait complete. 2024-04-17T08:00:33.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.345+0000 7f83c524c740 20 client.15864 trim_cache size 0 max 16384 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: "7e00b3d8-0760-4141-8373-52eadfa3def0": { 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "7e00b3d8-0760-4141-8373-52eadfa3def0", 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:00:33.349 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:00:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83a00a3bb0 msgr2=0x7f83a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83a00a3bb0 0x7f83a00a6070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f83b4005e30 tx=0x7f83b4032590 comp rx=0 tx=0).stop 2024-04-17T08:00:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1095d0 msgr2=0x7f83bc19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:00:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1095d0 0x7f83bc19ebd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e9f0 tx=0x7f83a400edf0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 shutdown_connections 2024-04-17T08:00:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f83a00a3bb0 0x7f83a00a6070 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc109f90 0x7f83bc1a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83bc1095d0 0x7f83bc19ebd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 --2- 172.21.15.46:0/3306776840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83bc1083e0 0x7f83bc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:00:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 >> 172.21.15.46:0/3306776840 conn(0x7f83bc0773a0 msgr2=0x7f83bc10c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:00:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 shutdown_connections 2024-04-17T08:00:33.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:00:33.346+0000 7f83c1000640 1 -- 172.21.15.46:0/3306776840 wait complete. 2024-04-17T08:00:33.365 INFO:tasks.cephfs.filesystem:scrub status for tag:7e00b3d8-0760-4141-8373-52eadfa3def0 - {'path': '/', 'tag': '7e00b3d8-0760-4141-8373-52eadfa3def0', 'options': 'recursive,force'} 2024-04-17T08:00:33.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:33 smithi046 ceph-mon[32038]: scrub queued for path: / 2024-04-17T08:00:33.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:33 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:33.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:33 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:00:33.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:33 smithi053 ceph-mon[36437]: scrub queued for path: / 2024-04-17T08:00:33.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:33 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:33.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:33 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:00:33.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:33 smithi062 ceph-mon[36507]: scrub queued for path: / 2024-04-17T08:00:33.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:33 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:33.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:33 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:00:34.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:34 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:00:34.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:34 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:34.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:34 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:34.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:34 smithi053 ceph-mon[36437]: pgmap v479: 129 pgs: 129 active+clean; 15 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-17T08:00:34.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:34 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:00:34.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:34 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:34.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:34 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:34.803 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:34 smithi062 ceph-mon[36507]: pgmap v479: 129 pgs: 129 active+clean; 15 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-17T08:00:34.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:34 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:00:34.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:34 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:34.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:34 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:00:34.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:34 smithi046 ceph-mon[32038]: pgmap v479: 129 pgs: 129 active+clean; 15 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-17T08:00:36.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:36 smithi053 ceph-mon[36437]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:00:36.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:36 smithi053 ceph-mon[36437]: pgmap v480: 129 pgs: 129 active+clean; 15 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-17T08:00:36.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:36 smithi046 ceph-mon[32038]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:00:36.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:36 smithi046 ceph-mon[32038]: pgmap v480: 129 pgs: 129 active+clean; 15 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-17T08:00:36.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:36 smithi062 ceph-mon[36507]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:00:36.802 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:36 smithi062 ceph-mon[36507]: pgmap v480: 129 pgs: 129 active+clean; 15 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-04-17T08:00:38.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:38 smithi053 ceph-mon[36437]: pgmap v481: 129 pgs: 129 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-04-17T08:00:39.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:38 smithi062 ceph-mon[36507]: pgmap v481: 129 pgs: 129 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-04-17T08:00:39.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:38 smithi046 ceph-mon[32038]: pgmap v481: 129 pgs: 129 active+clean; 18 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-04-17T08:00:40.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:40 smithi053 ceph-mon[36437]: pgmap v482: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 346 KiB/s wr, 19 op/s 2024-04-17T08:00:40.971 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:40 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:00:41.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:40 smithi062 ceph-mon[36507]: pgmap v482: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 346 KiB/s wr, 19 op/s 2024-04-17T08:00:41.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:40 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:00:41.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:40 smithi046 ceph-mon[32038]: pgmap v482: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 346 KiB/s wr, 19 op/s 2024-04-17T08:00:41.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:40 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:00:41.970 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:41 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:00:42.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:41 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:00:42.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:41 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:00:43.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:42 smithi062 ceph-mon[36507]: pgmap v483: 129 pgs: 129 active+clean; 20 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 585 KiB/s wr, 31 op/s 2024-04-17T08:00:43.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:42 smithi046 ceph-mon[32038]: pgmap v483: 129 pgs: 129 active+clean; 20 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 585 KiB/s wr, 31 op/s 2024-04-17T08:00:43.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:42 smithi053 ceph-mon[36437]: pgmap v483: 129 pgs: 129 active+clean; 20 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 585 KiB/s wr, 31 op/s 2024-04-17T08:00:45.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:44 smithi062 ceph-mon[36507]: pgmap v484: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 597 KiB/s wr, 33 op/s 2024-04-17T08:00:45.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:44 smithi046 ceph-mon[32038]: pgmap v484: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 597 KiB/s wr, 33 op/s 2024-04-17T08:00:45.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:44 smithi053 ceph-mon[36437]: pgmap v484: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 597 KiB/s wr, 33 op/s 2024-04-17T08:00:47.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:46 smithi062 ceph-mon[36507]: pgmap v485: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 533 KiB/s wr, 29 op/s 2024-04-17T08:00:47.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:46 smithi046 ceph-mon[32038]: pgmap v485: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 533 KiB/s wr, 29 op/s 2024-04-17T08:00:47.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:46 smithi053 ceph-mon[36437]: pgmap v485: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 533 KiB/s wr, 29 op/s 2024-04-17T08:00:47.978 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 11% (488/4290) Updating files: 11% (490/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 19% (828/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 33% (1444/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 45% (1966/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 55% (2372/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 76% (3265/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 88% (3787/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 96% (4148/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-17T08:00:48.003 INFO:tasks.workunit.client.0.smithi046.stderr:+ cd xfstests-dev 2024-04-17T08:00:48.003 INFO:tasks.workunit.client.0.smithi046.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-04-17T08:00:48.859 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:48 smithi046 ceph-mon[32038]: pgmap v486: 129 pgs: 129 active+clean; 20 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 579 KiB/s wr, 33 op/s 2024-04-17T08:00:49.022 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:48 smithi053 ceph-mon[36437]: pgmap v486: 129 pgs: 129 active+clean; 20 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 579 KiB/s wr, 33 op/s 2024-04-17T08:00:49.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:48 smithi062 ceph-mon[36507]: pgmap v486: 129 pgs: 129 active+clean; 20 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 579 KiB/s wr, 33 op/s 2024-04-17T08:00:51.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:50 smithi062 ceph-mon[36507]: pgmap v487: 129 pgs: 129 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 500 KiB/s wr, 30 op/s 2024-04-17T08:00:51.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:50 smithi046 ceph-mon[32038]: pgmap v487: 129 pgs: 129 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 500 KiB/s wr, 30 op/s 2024-04-17T08:00:51.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:50 smithi053 ceph-mon[36437]: pgmap v487: 129 pgs: 129 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 500 KiB/s wr, 30 op/s 2024-04-17T08:00:53.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:52 smithi062 ceph-mon[36507]: pgmap v488: 129 pgs: 129 active+clean; 25 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 764 KiB/s wr, 40 op/s 2024-04-17T08:00:53.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:52 smithi046 ceph-mon[32038]: pgmap v488: 129 pgs: 129 active+clean; 25 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 764 KiB/s wr, 40 op/s 2024-04-17T08:00:53.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:52 smithi053 ceph-mon[36437]: pgmap v488: 129 pgs: 129 active+clean; 25 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 764 KiB/s wr, 40 op/s 2024-04-17T08:00:55.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:54 smithi062 ceph-mon[36507]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 592 KiB/s wr, 43 op/s 2024-04-17T08:00:55.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:54 smithi046 ceph-mon[32038]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 592 KiB/s wr, 43 op/s 2024-04-17T08:00:55.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:54 smithi053 ceph-mon[36437]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 592 KiB/s wr, 43 op/s 2024-04-17T08:00:57.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:56 smithi062 ceph-mon[36507]: pgmap v490: 129 pgs: 129 active+clean; 31 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-17T08:00:57.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:56 smithi046 ceph-mon[32038]: pgmap v490: 129 pgs: 129 active+clean; 31 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-17T08:00:57.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:56 smithi053 ceph-mon[36437]: pgmap v490: 129 pgs: 129 active+clean; 31 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-17T08:00:59.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:00:58 smithi062 ceph-mon[36507]: pgmap v491: 129 pgs: 129 active+clean; 35 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-04-17T08:00:59.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:00:58 smithi046 ceph-mon[32038]: pgmap v491: 129 pgs: 129 active+clean; 35 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-04-17T08:00:59.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:00:58 smithi053 ceph-mon[36437]: pgmap v491: 129 pgs: 129 active+clean; 35 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-04-17T08:01:01.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:00 smithi062 ceph-mon[36507]: pgmap v492: 129 pgs: 129 active+clean; 36 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-17T08:01:01.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:00 smithi046 ceph-mon[32038]: pgmap v492: 129 pgs: 129 active+clean; 36 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-17T08:01:01.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:00 smithi053 ceph-mon[36437]: pgmap v492: 129 pgs: 129 active+clean; 36 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-17T08:01:03.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:02 smithi062 ceph-mon[36507]: pgmap v493: 129 pgs: 129 active+clean; 42 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 415 op/s 2024-04-17T08:01:03.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:02 smithi046 ceph-mon[32038]: pgmap v493: 129 pgs: 129 active+clean; 42 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 415 op/s 2024-04-17T08:01:03.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:02 smithi053 ceph-mon[36437]: pgmap v493: 129 pgs: 129 active+clean; 42 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 415 op/s 2024-04-17T08:01:03.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:01:03.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:01:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 -- 172.21.15.46:0/1058422636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0101de0 msgr2=0x7f66f0102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 --2- 172.21.15.46:0/1058422636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0101de0 0x7f66f0102200 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009ca0 tx=0x7f66e402c100 comp rx=0 tx=0).stop 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 -- 172.21.15.46:0/1058422636 shutdown_connections 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 --2- 172.21.15.46:0/1058422636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0103990 0x7f66f010a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 --2- 172.21.15.46:0/1058422636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f0102fd0 0x7f66f0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 --2- 172.21.15.46:0/1058422636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0101de0 0x7f66f0102200 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 -- 172.21.15.46:0/1058422636 >> 172.21.15.46:0/1058422636 conn(0x7f66f00fd370 msgr2=0x7f66f00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 -- 172.21.15.46:0/1058422636 shutdown_connections 2024-04-17T08:01:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.587+0000 7f66f6a00640 1 -- 172.21.15.46:0/1058422636 wait complete. 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.588+0000 7f66f6a00640 1 Processor -- start 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f6a00640 1 -- start start 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f0101de0 0x7f66f019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f6a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0102fd0 0x7f66f019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0103990 0x7f66f01a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f6a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66f010da30 con 0x7f66f0101de0 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f6a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66f010d8b0 con 0x7f66f0102fd0 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f6a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66f010dbb0 con 0x7f66f0103990 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f0101de0 0x7f66f019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f0101de0 0x7f66f019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44968/0 (socket says 172.21.15.46:44968) 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f5600640 1 -- 172.21.15.46:0/151241243 learned_addr learned my addr 172.21.15.46:0/151241243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.589+0000 7f66f4c00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0102fd0 0x7f66f019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f6000640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0103990 0x7f66f01a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f4c00640 1 -- 172.21.15.46:0/151241243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0103990 msgr2=0x7f66f01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f4c00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0103990 0x7f66f01a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f4c00640 1 -- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f0101de0 msgr2=0x7f66f019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f4c00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f0101de0 0x7f66f019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f4c00640 1 -- 172.21.15.46:0/151241243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f01a1f60 con 0x7f66f0102fd0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f4c00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0102fd0 0x7f66f019a8d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00e9f0 tx=0x7f66dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66ee000640 1 -- 172.21.15.46:0/151241243 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc00cd80 con 0x7f66f0102fd0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66ee000640 1 -- 172.21.15.46:0/151241243 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66dc0035f0 con 0x7f66f0102fd0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66ee000640 1 -- 172.21.15.46:0/151241243 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc010690 con 0x7f66f0102fd0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.590+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f01a2250 con 0x7f66f0102fd0 2024-04-17T08:01:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.591+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f01a2750 con 0x7f66f0102fd0 2024-04-17T08:01:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.594+0000 7f66fae16740 1 Processor -- start 2024-04-17T08:01:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.594+0000 7f66fae16740 1 -- start start 2024-04-17T08:01:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66ee000640 1 -- 172.21.15.46:0/151241243 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f66dc003b30 con 0x7f66f0102fd0 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66fae16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14318790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66fae16740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c1431d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66fae16740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14320360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66fae16740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c1427f120 con 0x558c14318d60 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66fae16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558c14225340 con 0x558c14318370 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66fae16740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c1427b0e0 con 0x558c1431ded0 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66f6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14318790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66f5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c1431d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66f6000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14318790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35532/0 (socket says 172.21.15.46:35532) 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66f6000640 1 -- 172.21.15.46:0/1898191766 learned_addr learned my addr 172.21.15.46:0/1898191766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.595+0000 7f66f4c00640 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14320360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66f6000640 1 -- 172.21.15.46:0/1898191766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 msgr2=0x558c14320360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66f6000640 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14320360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66f6000640 1 -- 172.21.15.46:0/1898191766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 msgr2=0x558c1431d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66f6000640 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c1431d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66f6000640 1 -- 172.21.15.46:0/1898191766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c1405b7d0 con 0x558c14318370 2024-04-17T08:01:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66f5600640 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c1431d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66f6000640 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14318790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009ca0 tx=0x7f66e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ac06949f1ec87 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66e2600640 1 -- 172.21.15.46:0/1898191766 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e801e070 con 0x558c14318370 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66e2600640 1 -- 172.21.15.46:0/1898191766 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e8009e50 con 0x558c14318370 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66e2600640 1 -- 172.21.15.46:0/1898191766 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e800eac0 con 0x558c14318370 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.596+0000 7f66ee000640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66d80a3c30 0x7f66d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.597+0000 7f66ee000640 1 -- 172.21.15.46:0/151241243 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f66dc0d0a30 con 0x7f66f0102fd0 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.597+0000 7f66f5600640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66d80a3c30 0x7f66d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.597+0000 7f66fae16740 1 -- 172.21.15.46:0/1898191766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 msgr2=0x558c14318790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.597+0000 7f66fae16740 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14318790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009ca0 tx=0x7f66e8011810 comp rx=0 tx=0).stop 2024-04-17T08:01:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 -- 172.21.15.46:0/1898191766 shutdown_connections 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14320360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c1431d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 --2- 172.21.15.46:0/1898191766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14318790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 -- 172.21.15.46:0/1898191766 >> 172.21.15.46:0/1898191766 conn(0x558c143138e0 msgr2=0x558c14313ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 -- 172.21.15.46:0/1898191766 shutdown_connections 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 -- 172.21.15.46:0/1898191766 wait complete. 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66f5600640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66d80a3c30 0x7f66d80a60f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66e4005e30 tx=0x7f66e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 -- start start 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.598+0000 7f66fae16740 1 Processor -- start 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66fae16740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318370 0x558c143228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66fae16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14322e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66fae16740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14329e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66fae16740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c14320e80 con 0x558c14318370 2024-04-17T08:01:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66fae16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558c14238710 con 0x558c14318d60 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66fae16740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c14321000 con 0x558c1431ded0 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66fae16740 10 client.0 _ll_register_callbacks cb 0x558c14233af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66f5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318370 0x558c143228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66f4c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14322e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66f5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318370 0x558c143228e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44974/0 (socket says 172.21.15.46:44974) 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66f5600640 1 -- 172.21.15.46:0/913063828 learned_addr learned my addr 172.21.15.46:0/913063828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.599+0000 7f66f6000640 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14329e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66f4c00640 1 -- 172.21.15.46:0/913063828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 msgr2=0x558c14329e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66f4c00640 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14329e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66f4c00640 1 -- 172.21.15.46:0/913063828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318370 msgr2=0x558c143228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66f4c00640 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318370 0x558c143228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66f4c00640 1 -- 172.21.15.46:0/913063828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c13f913f0 con 0x558c14318d60 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66f4c00640 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14322e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66dc01f000 tx=0x7f66dc098cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5494ef7ca9c30c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66d6a00640 1 -- 172.21.15.46:0/913063828 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc009450 con 0x558c14318d60 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e8009e50 con 0x558c14318d60 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66d6a00640 1 -- 172.21.15.46:0/913063828 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f66dc098f90 con 0x558c14318d60 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66d6a00640 1 -- 172.21.15.46:0/913063828 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66dc09d020 con 0x558c14318d60 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66fae16740 10 client.26027 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.600+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c13f510f0 con 0x558c14318d60 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66d6a00640 1 -- 172.21.15.46:0/913063828 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e8009e50 con 0x558c14318d60 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66d6a00640 1 -- 172.21.15.46:0/913063828 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f66dc0870b0 con 0x558c14318d60 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66fae16740 10 client.26027 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66fae16740 10 client.26027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66fae16740 20 client.26027 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66fae16740 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1432e6f0 0x558c14330b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66fae16740 4 client.26027 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c1428cb40 con 0x558c1432e6f0 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.601+0000 7f66f5600640 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1432e6f0 0x558c14330b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.602+0000 7f66f5600640 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1432e6f0 0x558c14330b70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a78c841557772d server_cookie=2b2a91a342d73ef in_seq=0 out_seq=0 2024-04-17T08:01:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.602+0000 7f66d6a00640 10 client.26027 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66d6a00640 1 -- 172.21.15.46:0/913063828 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c1428cb40 con 0x558c1432e6f0 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66d6a00640 10 client.26027 handle_command_reply: tid=0 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 client.26027 shutdown 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 msgr2=0x558c14322e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14322e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66dc01f000 tx=0x7f66dc098cf0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 shutdown_connections 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1432e6f0 0x558c14330b70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14329e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14322e20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 --2- 172.21.15.46:0/913063828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318370 0x558c143228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 >> 172.21.15.46:0/913063828 conn(0x558c143138e0 msgr2=0x558c14312ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66d6a00640 0 client.26027 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 shutdown_connections 2024-04-17T08:01:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 1 -- 172.21.15.46:0/913063828 wait complete. 2024-04-17T08:01:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.603+0000 7f66fae16740 20 client.26027 trim_cache size 0 max 16384 2024-04-17T08:01:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.624+0000 7f66fae16740 1 Processor -- start 2024-04-17T08:01:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66fae16740 1 -- start start 2024-04-17T08:01:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66fae16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14317250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66fae16740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c14329e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66fae16740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14335470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66fae16740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01af580 con 0x558c14318d60 2024-04-17T08:01:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66fae16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01ace40 con 0x558c14318370 2024-04-17T08:01:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66fae16740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01aac90 con 0x558c1431ded0 2024-04-17T08:01:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66f5600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14317250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66f4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c14329e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66f5600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14317250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35544/0 (socket says 172.21.15.46:35544) 2024-04-17T08:01:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66f5600640 1 -- 172.21.15.46:0/300910782 learned_addr learned my addr 172.21.15.46:0/300910782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.625+0000 7f66f6000640 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14335470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66f5600640 1 -- 172.21.15.46:0/300910782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 msgr2=0x558c14335470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66f5600640 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14335470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66f5600640 1 -- 172.21.15.46:0/300910782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 msgr2=0x558c14329e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66f5600640 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c14329e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66f5600640 1 -- 172.21.15.46:0/300910782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c143311f0 con 0x558c14318370 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66f5600640 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14317250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66e40325f0 tx=0x7f66e4041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=549b19ead31a1c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66e1c00640 1 -- 172.21.15.46:0/300910782 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4039040 con 0x558c14318370 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66e1c00640 1 -- 172.21.15.46:0/300910782 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e4032830 con 0x558c14318370 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.626+0000 7f66e1c00640 1 -- 172.21.15.46:0/300910782 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4052460 con 0x558c14318370 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 -- 172.21.15.46:0/300910782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 msgr2=0x558c14317250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14317250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66e40325f0 tx=0x7f66e4041000 comp rx=0 tx=0).stop 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 -- 172.21.15.46:0/300910782 shutdown_connections 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c1431ded0 0x558c14335470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c14318d60 0x558c14329e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 --2- 172.21.15.46:0/300910782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318370 0x558c14317250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 -- 172.21.15.46:0/300910782 >> 172.21.15.46:0/300910782 conn(0x558c143138e0 msgr2=0x558c14316b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 -- 172.21.15.46:0/300910782 shutdown_connections 2024-04-17T08:01:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.627+0000 7f66fae16740 1 -- 172.21.15.46:0/300910782 wait complete. 2024-04-17T08:01:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 -- start start 2024-04-17T08:01:03.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 Processor -- start 2024-04-17T08:01:03.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c14318370 0x558c14327fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14328500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1431ded0 0x558c14329350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01ab300 con 0x558c1431ded0 2024-04-17T08:01:03.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01ac770 con 0x558c14318d60 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01ab9d0 con 0x558c14318370 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66fae16740 10 client.0 _ll_register_callbacks cb 0x7f66dc003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66f4c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14328500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66f4c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14328500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35548/0 (socket says 172.21.15.46:35548) 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1431ded0 0x558c14329350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.628+0000 7f66f4c00640 1 -- 172.21.15.46:0/1876597400 learned_addr learned my addr 172.21.15.46:0/1876597400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f5600640 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c14318370 0x558c14327fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f4c00640 1 -- 172.21.15.46:0/1876597400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c14318370 msgr2=0x558c14327fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f4c00640 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c14318370 0x558c14327fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f4c00640 1 -- 172.21.15.46:0/1876597400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1431ded0 msgr2=0x558c14329350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f4c00640 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1431ded0 0x558c14329350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f4c00640 1 -- 172.21.15.46:0/1876597400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c1432e4d0 con 0x558c14318d60 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f6000640 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1431ded0 0x558c14329350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:01:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66f4c00640 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14328500 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009230 tx=0x7f66dc0039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb19f90fa02c6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66d6a00640 1 -- 172.21.15.46:0/1876597400 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc009430 con 0x558c14318d60 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66d6a00640 1 -- 172.21.15.46:0/1876597400 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f66dc0986c0 con 0x558c14318d60 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e4032830 con 0x558c14318d60 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66d6a00640 1 -- 172.21.15.46:0/1876597400 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66dc0a5590 con 0x558c14318d60 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.629+0000 7f66fae16740 10 client.26033 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c1439dd60 con 0x558c14318d60 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66d6a00640 1 -- 172.21.15.46:0/1876597400 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e4032830 con 0x558c14318d60 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66d6a00640 1 -- 172.21.15.46:0/1876597400 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f66dc09b7e0 con 0x558c14318d60 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66fae16740 10 client.26033 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66fae16740 10 client.26033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66fae16740 20 client.26033 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66fae16740 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1439e5b0 0x558c143a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66fae16740 4 client.26033 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:01:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c142869b0 con 0x558c1439e5b0 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.630+0000 7f66f5600640 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1439e5b0 0x558c143a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66f5600640 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1439e5b0 0x558c143a09d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c75488af60646c89 server_cookie=270a835baa42764b in_seq=0 out_seq=0 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66d6a00640 10 client.26033 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66d6a00640 1 -- 172.21.15.46:0/1876597400 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558c142869b0 con 0x558c1439e5b0 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66d6a00640 10 client.26033 handle_command_reply: tid=0 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 client.26033 shutdown 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 msgr2=0x558c14328500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14328500 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009230 tx=0x7f66dc0039d0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 shutdown_connections 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x558c1439e5b0 0x558c143a09d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c1431ded0 0x558c14329350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558c14318d60 0x558c14328500 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 --2- 172.21.15.46:0/1876597400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c14318370 0x558c14327fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66d6a00640 0 client.26033 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.631+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 >> 172.21.15.46:0/1876597400 conn(0x558c143138e0 msgr2=0x558c143173d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.632+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 shutdown_connections 2024-04-17T08:01:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.632+0000 7f66fae16740 1 -- 172.21.15.46:0/1876597400 wait complete. 2024-04-17T08:01:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.632+0000 7f66fae16740 20 client.26033 trim_cache size 0 max 16384 2024-04-17T08:01:03.635 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:01:03.635 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-17T08:01:03.635 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-17T08:01:03.635 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:01:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.632+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66d80a3c30 msgr2=0x7f66d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.632+0000 7f66f6a00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66d80a3c30 0x7f66d80a60f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66e4005e30 tx=0x7f66e4005fd0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.632+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0102fd0 msgr2=0x7f66f019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.632+0000 7f66f6a00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0102fd0 0x7f66f019a8d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00e9f0 tx=0x7f66dc00eeb0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 shutdown_connections 2024-04-17T08:01:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66d80a3c30 0x7f66d80a60f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0103990 0x7f66f01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f0102fd0 0x7f66f019a8d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 --2- 172.21.15.46:0/151241243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f0101de0 0x7f66f019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 >> 172.21.15.46:0/151241243 conn(0x7f66f00fd370 msgr2=0x7f66f0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 shutdown_connections 2024-04-17T08:01:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.633+0000 7f66f6a00640 1 -- 172.21.15.46:0/151241243 wait complete. 2024-04-17T08:01:03.654 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T08:01:03.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T08:01:03.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 -- 172.21.15.46:0/1973991208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a600749c0 msgr2=0x7f9a60074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 --2- 172.21.15.46:0/1973991208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a600749c0 0x7f9a60074e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009900 tx=0x7f9a4802bc30 comp rx=0 tx=0).stop 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 -- 172.21.15.46:0/1973991208 shutdown_connections 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 --2- 172.21.15.46:0/1973991208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a60109d70 0x7f9a6010c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 --2- 172.21.15.46:0/1973991208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a600749c0 0x7f9a60074e40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 --2- 172.21.15.46:0/1973991208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a60075fa0 0x7f9a600743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 -- 172.21.15.46:0/1973991208 >> 172.21.15.46:0/1973991208 conn(0x7f9a600fd420 msgr2=0x7f9a600ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 -- 172.21.15.46:0/1973991208 shutdown_connections 2024-04-17T08:01:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.872+0000 7f9a65000640 1 -- 172.21.15.46:0/1973991208 wait complete. 2024-04-17T08:01:03.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 Processor -- start 2024-04-17T08:01:03.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 -- start start 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600749c0 0x7f9a6019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60075fa0 0x7f9a6019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a60109d70 0x7f9a601a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6010fd40 con 0x7f9a600749c0 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6010fbc0 con 0x7f9a60109d70 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.874+0000 7f9a65000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6010fec0 con 0x7f9a60075fa0 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60075fa0 0x7f9a6019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600749c0 0x7f9a6019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60075fa0 0x7f9a6019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36530/0 (socket says 172.21.15.46:36530) 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a60109d70 0x7f9a601a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 -- 172.21.15.46:0/2523185122 learned_addr learned my addr 172.21.15.46:0/2523185122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 -- 172.21.15.46:0/2523185122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a60109d70 msgr2=0x7f9a601a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a60109d70 0x7f9a601a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 -- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600749c0 msgr2=0x7f9a6019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600749c0 0x7f9a6019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5ea00640 1 -- 172.21.15.46:0/2523185122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a601a1fd0 con 0x7f9a60075fa0 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.875+0000 7f9a5fe00640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a60109d70 0x7f9a601a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.876+0000 7f9a5ea00640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60075fa0 0x7f9a6019a940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a48000c00 tx=0x7f9a4802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.876+0000 7f9a57e00640 1 -- 172.21.15.46:0/2523185122 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48043020 con 0x7f9a60075fa0 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.876+0000 7f9a57e00640 1 -- 172.21.15.46:0/2523185122 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a480054e0 con 0x7f9a60075fa0 2024-04-17T08:01:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.876+0000 7f9a57e00640 1 -- 172.21.15.46:0/2523185122 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48003cd0 con 0x7f9a60075fa0 2024-04-17T08:01:03.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.876+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a601a2260 con 0x7f9a60075fa0 2024-04-17T08:01:03.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.876+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a601a2760 con 0x7f9a60075fa0 2024-04-17T08:01:03.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.878+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a60108aa0 con 0x7f9a60075fa0 2024-04-17T08:01:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.880+0000 7f9a57e00640 1 -- 172.21.15.46:0/2523185122 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a48046020 con 0x7f9a60075fa0 2024-04-17T08:01:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.882+0000 7f9a57e00640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a440a3c20 0x7f9a440a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.882+0000 7f9a57e00640 1 -- 172.21.15.46:0/2523185122 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9a480f2b70 con 0x7f9a60075fa0 2024-04-17T08:01:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.882+0000 7f9a5f400640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a440a3c20 0x7f9a440a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.883+0000 7f9a5f400640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a440a3c20 0x7f9a440a60e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009bd0 tx=0x7f9a50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:03.884+0000 7f9a57e00640 1 -- 172.21.15.46:0/2523185122 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9a4804d020 con 0x7f9a60075fa0 2024-04-17T08:01:04.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.228+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a6002d820 con 0x7f9a60075fa0 2024-04-17T08:01:04.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.230+0000 7f9a57e00640 1 -- 172.21.15.46:0/2523185122 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11872 (secure 0 0 0) 0x7f9a480bf160 con 0x7f9a60075fa0 2024-04-17T08:01:04.231 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:01:04.231 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T08:00:35.051221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T08:01:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a440a3c20 msgr2=0x7f9a440a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a440a3c20 0x7f9a440a60e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009bd0 tx=0x7f9a50005f90 comp rx=0 tx=0).stop 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60075fa0 msgr2=0x7f9a6019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60075fa0 0x7f9a6019a940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a48000c00 tx=0x7f9a4802c650 comp rx=0 tx=0).stop 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 shutdown_connections 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9a440a3c20 0x7f9a440a60e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a60109d70 0x7f9a601a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a60075fa0 0x7f9a6019a940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 --2- 172.21.15.46:0/2523185122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a600749c0 0x7f9a6019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.232+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 >> 172.21.15.46:0/2523185122 conn(0x7f9a600fd420 msgr2=0x7f9a60105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.233+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 shutdown_connections 2024-04-17T08:01:04.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.233+0000 7f9a65000640 1 -- 172.21.15.46:0/2523185122 wait complete. 2024-04-17T08:01:04.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T08:01:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 -- 172.21.15.46:0/3078073266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41053b0 msgr2=0x7ff4f4105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3078073266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41053b0 0x7ff4f4105830 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009900 tx=0x7ff4e002bc30 comp rx=0 tx=0).stop 2024-04-17T08:01:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 -- 172.21.15.46:0/3078073266 shutdown_connections 2024-04-17T08:01:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3078073266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f4105d70 0x7ff4f410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3078073266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41053b0 0x7ff4f4105830 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3078073266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41041c0 0x7ff4f41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 -- 172.21.15.46:0/3078073266 >> 172.21.15.46:0/3078073266 conn(0x7ff4f40ff750 msgr2=0x7ff4f4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 -- 172.21.15.46:0/3078073266 shutdown_connections 2024-04-17T08:01:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.471+0000 7ff4fb800640 1 -- 172.21.15.46:0/3078073266 wait complete. 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 Processor -- start 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 -- start start 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41041c0 0x7ff4f419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41053b0 0x7ff4f419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f4105d70 0x7ff4f41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410fea0 con 0x7ff4f41041c0 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410fd20 con 0x7ff4f4105d70 2024-04-17T08:01:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.472+0000 7ff4fb800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4110020 con 0x7ff4f41053b0 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41053b0 0x7ff4f419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f4105d70 0x7ff4f41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41041c0 0x7ff4f419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f4105d70 0x7ff4f41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35602/0 (socket says 172.21.15.46:35602) 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41053b0 0x7ff4f419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36550/0 (socket says 172.21.15.46:36550) 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f9800640 1 -- 172.21.15.46:0/3402021415 learned_addr learned my addr 172.21.15.46:0/3402021415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f3e00640 1 -- 172.21.15.46:0/3402021415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f4105d70 msgr2=0x7ff4f41a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f4105d70 0x7ff4f41a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f3e00640 1 -- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41041c0 msgr2=0x7ff4f419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41041c0 0x7ff4f419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.473+0000 7ff4f3e00640 1 -- 172.21.15.46:0/3402021415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f41a20d0 con 0x7ff4f41053b0 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.474+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41053b0 0x7ff4f419aa90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0036040 tx=0x7ff4e002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.474+0000 7ff4f1600640 1 -- 172.21.15.46:0/3402021415 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0043020 con 0x7ff4f41053b0 2024-04-17T08:01:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.474+0000 7ff4f1600640 1 -- 172.21.15.46:0/3402021415 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e00054e0 con 0x7ff4f41053b0 2024-04-17T08:01:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.474+0000 7ff4f1600640 1 -- 172.21.15.46:0/3402021415 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0003c30 con 0x7ff4f41053b0 2024-04-17T08:01:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.474+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41a2360 con 0x7ff4f41053b0 2024-04-17T08:01:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.474+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41a28d0 con 0x7ff4f41053b0 2024-04-17T08:01:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.477+0000 7ff4fe610740 1 Processor -- start 2024-04-17T08:01:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4fe610740 1 -- start start 2024-04-17T08:01:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4f1600640 1 -- 172.21.15.46:0/3402021415 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff4e0046020 con 0x7ff4f41053b0 2024-04-17T08:01:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af91fa0 0x55671af923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671af97490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671af99f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4fe610740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55671aef8020 con 0x55671af91fa0 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4fe610740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55671ae9e460 con 0x55671af92990 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4fe610740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55671aef3fe0 con 0x55671af97b00 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4f8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671af97490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4f8e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671af97490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35606/0 (socket says 172.21.15.46:35606) 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af91fa0 0x55671af923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.478+0000 7ff4f8e00640 1 -- 172.21.15.46:0/740574790 learned_addr learned my addr 172.21.15.46:0/740574790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671af99f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f8e00640 1 -- 172.21.15.46:0/740574790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 msgr2=0x55671af99f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671af99f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f8e00640 1 -- 172.21.15.46:0/740574790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af91fa0 msgr2=0x55671af923c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af91fa0 0x55671af923c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f8e00640 1 -- 172.21.15.46:0/740574790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55671aea6040 con 0x55671af92990 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f9800640 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af91fa0 0x55671af923c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.479+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671af97490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009ca0 tx=0x7ff4e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47df6c5e1b0abd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4e6000640 1 -- 172.21.15.46:0/740574790 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004220 con 0x55671af92990 2024-04-17T08:01:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4e6000640 1 -- 172.21.15.46:0/740574790 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e8009e50 con 0x55671af92990 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4e6000640 1 -- 172.21.15.46:0/740574790 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80197a0 con 0x55671af92990 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4f1600640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4dc0a3c30 0x7ff4dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4f1600640 1 -- 172.21.15.46:0/3402021415 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ff4e00f2710 con 0x7ff4f41053b0 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4f9800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4dc0a3c30 0x7ff4dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4fe610740 1 -- 172.21.15.46:0/740574790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 msgr2=0x55671af97490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.480+0000 7ff4fe610740 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671af97490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009ca0 tx=0x7ff4e8011810 comp rx=0 tx=0).stop 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 -- 172.21.15.46:0/740574790 shutdown_connections 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671af99f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671af97490 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 --2- 172.21.15.46:0/740574790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af91fa0 0x55671af923c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 -- 172.21.15.46:0/740574790 >> 172.21.15.46:0/740574790 conn(0x55671af8d510 msgr2=0x55671af8d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 -- 172.21.15.46:0/740574790 shutdown_connections 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4f9800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4dc0a3c30 0x7ff4dc0a60f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55671af931f0 tx=0x7ff4ec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 -- 172.21.15.46:0/740574790 wait complete. 2024-04-17T08:01:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.481+0000 7ff4fe610740 1 -- start start 2024-04-17T08:01:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 1 Processor -- start 2024-04-17T08:01:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af91fa0 0x55671af9c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af92990 0x55671af9ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671afa3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55671af9aab0 con 0x55671af92990 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55671aeb1830 con 0x55671af91fa0 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55671af9ac30 con 0x55671af97b00 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.482+0000 7ff4fe610740 10 client.0 _ll_register_callbacks cb 0x55671ae7ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671afa3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af92990 0x55671af9ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671afa3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36564/0 (socket says 172.21.15.46:36564) 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 -- 172.21.15.46:0/3076549773 learned_addr learned my addr 172.21.15.46:0/3076549773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af91fa0 0x55671af9c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 -- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af91fa0 msgr2=0x55671af9c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af91fa0 0x55671af9c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 -- 172.21.15.46:0/3076549773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af92990 msgr2=0x55671af9ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af92990 0x55671af9ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 -- 172.21.15.46:0/3076549773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671aeb5920 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4f9800640 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671afa3a70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009a80 tx=0x7ff4ec004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d040a97a496e1304 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.483+0000 7ff4daa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.484+0000 7ff4daa00640 1 -- 172.21.15.46:0/3076549773 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec012df0 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.484+0000 7ff4daa00640 1 -- 172.21.15.46:0/3076549773 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ff4ec018030 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.484+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e8009e50 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.484+0000 7ff4daa00640 1 -- 172.21.15.46:0/3076549773 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec034050 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.484+0000 7ff4fe610740 10 client.26029 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.484+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55671ae837f0 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.484+0000 7ff4daa00640 1 -- 172.21.15.46:0/3076549773 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e8009e50 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4daa00640 1 -- 172.21.15.46:0/3076549773 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7ff4ec0050b0 con 0x55671af97b00 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4fe610740 10 client.26029 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4fe610740 10 client.26029 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:01:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4fe610740 20 client.26029 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4fe610740 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671afa8080 0x55671afaa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4fe610740 4 client.26029 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55671af05a40 con 0x55671afa8080 2024-04-17T08:01:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.485+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671afa8080 0x55671afaa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.486+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671afa8080 0x55671afaa4e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82d32402dc7801f5 server_cookie=4069fea8dbf51ee0 in_seq=0 out_seq=0 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.486+0000 7ff4daa00640 10 client.26029 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4daa00640 1 -- 172.21.15.46:0/3076549773 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55671af05a40 con 0x55671afa8080 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4daa00640 10 client.26029 handle_command_reply: tid=0 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 client.26029 shutdown 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 msgr2=0x55671afa3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671afa3a70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009a80 tx=0x7ff4ec004ab0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 shutdown_connections 2024-04-17T08:01:04.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671afa8080 0x55671afaa4e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af97b00 0x55671afa3a70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af92990 0x55671af9ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 --2- 172.21.15.46:0/3076549773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af91fa0 0x55671af9c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4daa00640 0 client.26029 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 >> 172.21.15.46:0/3076549773 conn(0x55671af8d510 msgr2=0x55671af8c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 shutdown_connections 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 1 -- 172.21.15.46:0/3076549773 wait complete. 2024-04-17T08:01:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.487+0000 7ff4fe610740 20 client.26029 trim_cache size 0 max 16384 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 Processor -- start 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 -- start start 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671af88570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671af90e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41af5e0 con 0x55671af97b00 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41acea0 con 0x55671af92990 2024-04-17T08:01:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4fe610740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41aacf0 con 0x55671af91fa0 2024-04-17T08:01:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.503+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35624/0 (socket says 172.21.15.46:35624) 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f3e00640 1 -- 172.21.15.46:0/410550293 learned_addr learned my addr 172.21.15.46:0/410550293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671af88570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f9800640 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671af90e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f3e00640 1 -- 172.21.15.46:0/410550293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 msgr2=0x55671af88570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671af88570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f3e00640 1 -- 172.21.15.46:0/410550293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 msgr2=0x55671af90e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671af90e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f3e00640 1 -- 172.21.15.46:0/410550293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55671ac0a3f0 con 0x55671af92990 2024-04-17T08:01:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f9800640 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671af90e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:01:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671af88570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.504+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3a30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00be300 tx=0x7ff4e00f8530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9cce6a3ef44595d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.505+0000 7ff4e5600640 1 -- 172.21.15.46:0/410550293 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00ba040 con 0x55671af92990 2024-04-17T08:01:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.505+0000 7ff4e5600640 1 -- 172.21.15.46:0/410550293 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e0005690 con 0x55671af92990 2024-04-17T08:01:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.505+0000 7ff4e5600640 1 -- 172.21.15.46:0/410550293 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00bed00 con 0x55671af92990 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.505+0000 7ff4fe610740 1 -- 172.21.15.46:0/410550293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 msgr2=0x55671afa3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.505+0000 7ff4fe610740 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3a30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00be300 tx=0x7ff4e00f8530 comp rx=0 tx=0).stop 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 -- 172.21.15.46:0/410550293 shutdown_connections 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671af90e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3a30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 --2- 172.21.15.46:0/410550293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671af88570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 -- 172.21.15.46:0/410550293 >> 172.21.15.46:0/410550293 conn(0x55671af8d510 msgr2=0x55671af90770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 -- 172.21.15.46:0/410550293 shutdown_connections 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 -- 172.21.15.46:0/410550293 wait complete. 2024-04-17T08:01:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.506+0000 7ff4fe610740 1 -- start start 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 1 Processor -- start 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671afa32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671afa1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41ab360 con 0x55671af97b00 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41ac7d0 con 0x55671af92990 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41aba30 con 0x55671af91fa0 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.507+0000 7ff4fe610740 10 client.0 _ll_register_callbacks cb 0x7ff4ec00fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671afa32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671afa1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671afa32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36582/0 (socket says 172.21.15.46:36582) 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 -- 172.21.15.46:0/1854302900 learned_addr learned my addr 172.21.15.46:0/1854302900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 -- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 msgr2=0x55671afa3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 -- 172.21.15.46:0/1854302900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 msgr2=0x55671afa1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671afa1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 -- 172.21.15.46:0/1854302900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55671acd47d0 con 0x55671af91fa0 2024-04-17T08:01:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4f8e00640 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671afa32c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009710 tx=0x7ff4e8005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6ff0e8bea3f4c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.508+0000 7ff4daa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.509+0000 7ff4daa00640 1 -- 172.21.15.46:0/1854302900 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8009780 con 0x55671af91fa0 2024-04-17T08:01:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.509+0000 7ff4daa00640 1 -- 172.21.15.46:0/1854302900 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7ff4e802cbc0 con 0x55671af91fa0 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.509+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55671aeff8b0 con 0x55671af91fa0 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.509+0000 7ff4daa00640 1 -- 172.21.15.46:0/1854302900 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4e803a050 con 0x55671af91fa0 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.509+0000 7ff4fe610740 10 client.26035 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.509+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55671b00edf0 con 0x55671af91fa0 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.509+0000 7ff4daa00640 1 -- 172.21.15.46:0/1854302900 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55671aeff8b0 con 0x55671af91fa0 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4daa00640 1 -- 172.21.15.46:0/1854302900 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7ff4e80051e0 con 0x55671af91fa0 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4fe610740 10 client.26035 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4fe610740 10 client.26035 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4fe610740 20 client.26035 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4fe610740 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671b00f600 0x55671b011a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4fe610740 4 client.26035 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55671ae7b570 con 0x55671b00f600 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671b00f600 0x55671b011a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4f3e00640 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671b00f600 0x55671b011a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89aae3163d9c3f1d server_cookie=65d30848ec3f70cb in_seq=0 out_seq=0 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.510+0000 7ff4daa00640 10 client.26035 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4daa00640 1 -- 172.21.15.46:0/1854302900 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55671ae7b570 con 0x55671b00f600 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4daa00640 10 client.26035 handle_command_reply: tid=0 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 client.26035 shutdown 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 msgr2=0x55671afa32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671afa32c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009710 tx=0x7ff4e8005db0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 shutdown_connections 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55671b00f600 0x55671b011a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55671af97b00 0x55671afa1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55671af92990 0x55671afa3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 --2- 172.21.15.46:0/1854302900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55671af91fa0 0x55671afa32c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4daa00640 0 client.26035 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 >> 172.21.15.46:0/1854302900 conn(0x55671af8d510 msgr2=0x55671af91000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 shutdown_connections 2024-04-17T08:01:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 1 -- 172.21.15.46:0/1854302900 wait complete. 2024-04-17T08:01:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.511+0000 7ff4fe610740 20 client.26035 trim_cache size 0 max 16384 2024-04-17T08:01:04.514 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4dc0a3c30 msgr2=0x7ff4dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4dc0a3c30 0x7ff4dc0a60f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55671af931f0 tx=0x7ff4ec009500 comp rx=0 tx=0).stop 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41053b0 msgr2=0x7ff4f419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41053b0 0x7ff4f419aa90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0036040 tx=0x7ff4e002c650 comp rx=0 tx=0).stop 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 shutdown_connections 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7ff4dc0a3c30 0x7ff4dc0a60f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f4105d70 0x7ff4f41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4f41053b0 0x7ff4f419aa90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 --2- 172.21.15.46:0/3402021415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f41041c0 0x7ff4f419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 >> 172.21.15.46:0/3402021415 conn(0x7ff4f40ff750 msgr2=0x7ff4f41086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 shutdown_connections 2024-04-17T08:01:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.512+0000 7ff4fb800640 1 -- 172.21.15.46:0/3402021415 wait complete. 2024-04-17T08:01:04.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T08:01:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.759+0000 7f9b14200640 1 -- 172.21.15.46:0/1086116987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1001e0 msgr2=0x7f9b0c100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.759+0000 7f9b14200640 1 --2- 172.21.15.46:0/1086116987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1001e0 0x7f9b0c100600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009860 tx=0x7f9af802bbb0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.760+0000 7f9b14200640 1 -- 172.21.15.46:0/1086116987 shutdown_connections 2024-04-17T08:01:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.760+0000 7f9b14200640 1 --2- 172.21.15.46:0/1086116987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c101d90 0x7f9b0c108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.760+0000 7f9b14200640 1 --2- 172.21.15.46:0/1086116987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b0c1013d0 0x7f9b0c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.760+0000 7f9b14200640 1 --2- 172.21.15.46:0/1086116987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1001e0 0x7f9b0c100600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.760+0000 7f9b14200640 1 -- 172.21.15.46:0/1086116987 >> 172.21.15.46:0/1086116987 conn(0x7f9b0c0fb750 msgr2=0x7f9b0c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.761+0000 7f9b14200640 1 -- 172.21.15.46:0/1086116987 shutdown_connections 2024-04-17T08:01:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.761+0000 7f9b14200640 1 -- 172.21.15.46:0/1086116987 wait complete. 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.763+0000 7f9b14200640 1 Processor -- start 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.764+0000 7f9b14200640 1 -- start start 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.764+0000 7f9b14200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b0c1001e0 0x7f9b0c111430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.764+0000 7f9b14200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1013d0 0x7f9b0c111970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.764+0000 7f9b14200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c101d90 0x7f9b0c1ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.764+0000 7f9b14200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10b620 con 0x7f9b0c101d90 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.764+0000 7f9b14200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10b4a0 con 0x7f9b0c1013d0 2024-04-17T08:01:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.764+0000 7f9b14200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c10b7a0 con 0x7f9b0c1001e0 2024-04-17T08:01:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1013d0 0x7f9b0c111970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1013d0 0x7f9b0c111970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35644/0 (socket says 172.21.15.46:35644) 2024-04-17T08:01:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b11800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b0c1001e0 0x7f9b0c111430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 -- 172.21.15.46:0/698274088 learned_addr learned my addr 172.21.15.46:0/698274088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b12200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c101d90 0x7f9b0c1ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 -- 172.21.15.46:0/698274088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b0c1001e0 msgr2=0x7f9b0c111430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b0c1001e0 0x7f9b0c111430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 -- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c101d90 msgr2=0x7f9b0c1ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c101d90 0x7f9b0c1ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.765+0000 7f9b10e00640 1 -- 172.21.15.46:0/698274088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c1aef20 con 0x7f9b0c1013d0 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.766+0000 7f9b10e00640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1013d0 0x7f9b0c111970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b000097d0 tx=0x7f9b00009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.766+0000 7f9b0a000640 1 -- 172.21.15.46:0/698274088 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00018020 con 0x7f9b0c1013d0 2024-04-17T08:01:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.766+0000 7f9b0a000640 1 -- 172.21.15.46:0/698274088 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0000ed90 con 0x7f9b0c1013d0 2024-04-17T08:01:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.766+0000 7f9b0a000640 1 -- 172.21.15.46:0/698274088 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00013370 con 0x7f9b0c1013d0 2024-04-17T08:01:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.766+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c1af0f0 con 0x7f9b0c1013d0 2024-04-17T08:01:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.766+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c1af5b0 con 0x7f9b0c1013d0 2024-04-17T08:01:04.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.776+0000 7f9b1702f740 1 Processor -- start 2024-04-17T08:01:04.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.776+0000 7f9b0a000640 1 -- 172.21.15.46:0/698274088 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b0001b020 con 0x7f9b0c1013d0 2024-04-17T08:01:04.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.777+0000 7f9b1702f740 1 -- start start 2024-04-17T08:01:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.778+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.778+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a7b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.778+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a7df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.778+0000 7f9b1702f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5591749dc020 con 0x559174a76990 2024-04-17T08:01:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.778+0000 7f9b1702f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559174982460 con 0x559174a7bb00 2024-04-17T08:01:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.778+0000 7f9b1702f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5591749d7fe0 con 0x559174a75fa0 2024-04-17T08:01:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.780+0000 7f9b10e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a7df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.780+0000 7f9b10e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a7df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35648/0 (socket says 172.21.15.46:35648) 2024-04-17T08:01:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.780+0000 7f9b10e00640 1 -- 172.21.15.46:0/743300652 learned_addr learned my addr 172.21.15.46:0/743300652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.780+0000 7f9b12200640 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.781+0000 7f9b11800640 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a7b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.781+0000 7f9b10e00640 1 -- 172.21.15.46:0/743300652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 msgr2=0x559174a763c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.781+0000 7f9b10e00640 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a763c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.781+0000 7f9b10e00640 1 -- 172.21.15.46:0/743300652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 msgr2=0x559174a7b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.781+0000 7f9b10e00640 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a7b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.781+0000 7f9b10e00640 1 -- 172.21.15.46:0/743300652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917498a040 con 0x559174a7bb00 2024-04-17T08:01:04.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.782+0000 7f9b10e00640 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a7df90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b000d8550 tx=0x7f9b000d8a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6516de6dec7045bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.782+0000 7f9b0a000640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9af40a3d00 0x7f9af40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.783+0000 7f9b0a000640 1 -- 172.21.15.46:0/698274088 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9b000d03d0 con 0x7f9b0c1013d0 2024-04-17T08:01:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.784+0000 7f9afea00640 1 -- 172.21.15.46:0/743300652 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000e4070 con 0x559174a7bb00 2024-04-17T08:01:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.785+0000 7f9afea00640 1 -- 172.21.15.46:0/743300652 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0009c9c0 con 0x559174a7bb00 2024-04-17T08:01:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.785+0000 7f9afea00640 1 -- 172.21.15.46:0/743300652 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0009bc20 con 0x559174a7bb00 2024-04-17T08:01:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.785+0000 7f9b12200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9af40a3d00 0x7f9af40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.786+0000 7f9b1702f740 1 -- 172.21.15.46:0/743300652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 msgr2=0x559174a7df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.786+0000 7f9b1702f740 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a7df90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b000d8550 tx=0x7f9b000d8a10 comp rx=0 tx=0).stop 2024-04-17T08:01:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.787+0000 7f9b1702f740 1 -- 172.21.15.46:0/743300652 shutdown_connections 2024-04-17T08:01:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.787+0000 7f9b1702f740 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a7df90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.787+0000 7f9b1702f740 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a7b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.787+0000 7f9b1702f740 1 --2- 172.21.15.46:0/743300652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.787+0000 7f9b1702f740 1 -- 172.21.15.46:0/743300652 >> 172.21.15.46:0/743300652 conn(0x559174a71510 msgr2=0x559174a71910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.788+0000 7f9b12200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9af40a3d00 0x7f9af40a61c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x559174a771f0 tx=0x7f9b04009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.788+0000 7f9b1702f740 1 -- 172.21.15.46:0/743300652 shutdown_connections 2024-04-17T08:01:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.788+0000 7f9b1702f740 1 -- 172.21.15.46:0/743300652 wait complete. 2024-04-17T08:01:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.790+0000 7f9b1702f740 1 -- start start 2024-04-17T08:01:04.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.791+0000 7f9b1702f740 1 Processor -- start 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a75fa0 0x559174a80510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a76990 0x559174a80a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a7bb00 0x559174a87a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b1702f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559174a7eb00 con 0x559174a75fa0 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b1702f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559174995830 con 0x559174a76990 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b1702f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559174a7ec80 con 0x559174a7bb00 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b1702f740 10 client.0 _ll_register_callbacks cb 0x559174963fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.793+0000 7f9b10e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a76990 0x559174a80a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b10e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a76990 0x559174a80a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35650/0 (socket says 172.21.15.46:35650) 2024-04-17T08:01:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b10e00640 1 -- 172.21.15.46:0/1807731055 learned_addr learned my addr 172.21.15.46:0/1807731055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b10e00640 1 -- 172.21.15.46:0/1807731055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a7bb00 msgr2=0x559174a87a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b11800640 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a75fa0 0x559174a80510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b10e00640 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a7bb00 0x559174a87a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b10e00640 1 -- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a75fa0 msgr2=0x559174a80510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b10e00640 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a75fa0 0x559174a80510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b10e00640 1 -- 172.21.15.46:0/1807731055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559174999920 con 0x559174a76990 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.794+0000 7f9b11800640 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a75fa0 0x559174a80510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.795+0000 7f9b10e00640 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a76990 0x559174a80a50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b000d8520 tx=0x7f9b0000a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0eff63bd081a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.795+0000 7f9af3400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.795+0000 7f9af3400640 1 -- 172.21.15.46:0/1807731055 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0009bc20 con 0x559174a76990 2024-04-17T08:01:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.795+0000 7f9af3400640 1 -- 172.21.15.46:0/1807731055 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9b0009a5c0 con 0x559174a76990 2024-04-17T08:01:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.795+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b0009c9c0 con 0x559174a76990 2024-04-17T08:01:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.796+0000 7f9af3400640 1 -- 172.21.15.46:0/1807731055 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b00098560 con 0x559174a76990 2024-04-17T08:01:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.796+0000 7f9b1702f740 10 client.26069 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.796+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5591749677f0 con 0x559174a76990 2024-04-17T08:01:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.797+0000 7f9af3400640 1 -- 172.21.15.46:0/1807731055 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0009c9c0 con 0x559174a76990 2024-04-17T08:01:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.797+0000 7f9af3400640 1 -- 172.21.15.46:0/1807731055 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f9b0009c520 con 0x559174a76990 2024-04-17T08:01:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.798+0000 7f9b1702f740 10 client.26069 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.798+0000 7f9b1702f740 10 client.26069 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:01:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.798+0000 7f9b1702f740 20 client.26069 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.798+0000 7f9b1702f740 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174a8df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.798+0000 7f9b1702f740 4 client.26069 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.798+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591749e9a40 con 0x559174a8bb80 2024-04-17T08:01:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.799+0000 7f9b11800640 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174a8df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.800+0000 7f9b11800640 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174a8df70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b46e02f27368075 server_cookie=e09d5ab272e905fc in_seq=0 out_seq=0 2024-04-17T08:01:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.800+0000 7f9af3400640 10 client.26069 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.803+0000 7f9af3400640 1 -- 172.21.15.46:0/1807731055 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591749e9a40 con 0x559174a8bb80 2024-04-17T08:01:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.803+0000 7f9af3400640 10 client.26069 handle_command_reply: tid=0 2024-04-17T08:01:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.803+0000 7f9b1702f740 1 client.26069 shutdown 2024-04-17T08:01:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a76990 msgr2=0x559174a80a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a76990 0x559174a80a50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b000d8520 tx=0x7f9b0000a250 comp rx=0 tx=0).stop 2024-04-17T08:01:04.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 shutdown_connections 2024-04-17T08:01:04.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a7bb00 0x559174a87a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a76990 0x559174a80a50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174a8df70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9af3400640 0 client.26069 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:04.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 --2- 172.21.15.46:0/1807731055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a75fa0 0x559174a80510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.804+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 >> 172.21.15.46:0/1807731055 conn(0x559174a71510 msgr2=0x559174a70710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.805+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 shutdown_connections 2024-04-17T08:01:04.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.805+0000 7f9b1702f740 1 -- 172.21.15.46:0/1807731055 wait complete. 2024-04-17T08:01:04.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.805+0000 7f9b1702f740 20 client.26069 trim_cache size 0 max 16384 2024-04-17T08:01:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.844+0000 7f9b1702f740 1 Processor -- start 2024-04-17T08:01:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.844+0000 7f9b1702f740 1 -- start start 2024-04-17T08:01:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.845+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a6fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.845+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.845+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.845+0000 7f9b1702f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1bc540 con 0x559174a76990 2024-04-17T08:01:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.845+0000 7f9b1702f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b9e00 con 0x559174a7bb00 2024-04-17T08:01:04.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.845+0000 7f9b1702f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b7c50 con 0x559174a75fa0 2024-04-17T08:01:04.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.846+0000 7f9b10e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.846+0000 7f9b12200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.846+0000 7f9b11800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a6fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.846+0000 7f9b12200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35666/0 (socket says 172.21.15.46:35666) 2024-04-17T08:01:04.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.846+0000 7f9b12200640 1 -- 172.21.15.46:0/4270113113 learned_addr learned my addr 172.21.15.46:0/4270113113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.847+0000 7f9b11800640 1 -- 172.21.15.46:0/4270113113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 msgr2=0x559174a703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.847+0000 7f9b11800640 1 --2- 172.21.15.46:0/4270113113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.847+0000 7f9b11800640 1 -- 172.21.15.46:0/4270113113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 msgr2=0x559174a879e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.847+0000 7f9b11800640 1 --2- 172.21.15.46:0/4270113113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a879e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.847+0000 7f9b11800640 1 -- 172.21.15.46:0/4270113113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591746ee3f0 con 0x559174a75fa0 2024-04-17T08:01:04.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.848+0000 7f9b11800640 1 --2- 172.21.15.46:0/4270113113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a6fe80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9af802c0f0 tx=0x7f9af8035b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296eb20b9f1c079f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.849+0000 7f9afe000640 1 -- 172.21.15.46:0/4270113113 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8039040 con 0x559174a75fa0 2024-04-17T08:01:04.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.849+0000 7f9afe000640 1 -- 172.21.15.46:0/4270113113 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af8043070 con 0x559174a75fa0 2024-04-17T08:01:04.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.849+0000 7f9afe000640 1 -- 172.21.15.46:0/4270113113 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af80053c0 con 0x559174a75fa0 2024-04-17T08:01:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 -- 172.21.15.46:0/4270113113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 msgr2=0x559174a6fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 --2- 172.21.15.46:0/4270113113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a6fe80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9af802c0f0 tx=0x7f9af8035b60 comp rx=0 tx=0).stop 2024-04-17T08:01:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 -- 172.21.15.46:0/4270113113 shutdown_connections 2024-04-17T08:01:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 --2- 172.21.15.46:0/4270113113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 --2- 172.21.15.46:0/4270113113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 --2- 172.21.15.46:0/4270113113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a6fe80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 -- 172.21.15.46:0/4270113113 >> 172.21.15.46:0/4270113113 conn(0x559174a71510 msgr2=0x559174a80510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.851+0000 7f9b1702f740 1 -- 172.21.15.46:0/4270113113 shutdown_connections 2024-04-17T08:01:04.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.852+0000 7f9b1702f740 1 -- 172.21.15.46:0/4270113113 wait complete. 2024-04-17T08:01:04.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.852+0000 7f9b1702f740 1 -- start start 2024-04-17T08:01:04.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.852+0000 7f9b1702f740 1 Processor -- start 2024-04-17T08:01:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.853+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a85600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b1702f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a86610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b1702f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b82c0 con 0x559174a76990 2024-04-17T08:01:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b1702f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b9730 con 0x559174a7bb00 2024-04-17T08:01:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b1702f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b8990 con 0x559174a75fa0 2024-04-17T08:01:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b1702f740 10 client.0 _ll_register_callbacks cb 0x7f9b000d83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b11800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b11800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a850c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36646/0 (socket says 172.21.15.46:36646) 2024-04-17T08:01:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b11800640 1 -- 172.21.15.46:0/2675641765 learned_addr learned my addr 172.21.15.46:0/2675641765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.854+0000 7f9b12200640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a86610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.855+0000 7f9b12200640 1 -- 172.21.15.46:0/2675641765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 msgr2=0x559174a850c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.855+0000 7f9b12200640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a850c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.855+0000 7f9b12200640 1 -- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 msgr2=0x559174a85600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.855+0000 7f9b10e00640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.856+0000 7f9b12200640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a85600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.856+0000 7f9b12200640 1 -- 172.21.15.46:0/2675641765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591747b87d0 con 0x559174a7bb00 2024-04-17T08:01:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.857+0000 7f9b10e00640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a85600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.857+0000 7f9b12200640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a86610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x559174a7e4a0 tx=0x7f9b04004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fecad10f6fa6e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.857+0000 7f9af3400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:04.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.858+0000 7f9af3400640 1 -- 172.21.15.46:0/2675641765 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04011cc0 con 0x559174a7bb00 2024-04-17T08:01:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.858+0000 7f9af3400640 1 -- 172.21.15.46:0/2675641765 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9b04021400 con 0x559174a7bb00 2024-04-17T08:01:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.858+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af8043070 con 0x559174a7bb00 2024-04-17T08:01:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.858+0000 7f9af3400640 1 -- 172.21.15.46:0/2675641765 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b04033050 con 0x559174a7bb00 2024-04-17T08:01:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.858+0000 7f9b1702f740 10 client.26075 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:04.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.858+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559174af1e70 con 0x559174a7bb00 2024-04-17T08:01:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.859+0000 7f9af3400640 1 -- 172.21.15.46:0/2675641765 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af8043070 con 0x559174a7bb00 2024-04-17T08:01:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.859+0000 7f9af3400640 1 -- 172.21.15.46:0/2675641765 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f9b0400f040 con 0x559174a7bb00 2024-04-17T08:01:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.860+0000 7f9b1702f740 10 client.26075 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.860+0000 7f9b1702f740 10 client.26075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:01:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.860+0000 7f9b1702f740 20 client.26075 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.860+0000 7f9b1702f740 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174af47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.861+0000 7f9b1702f740 4 client.26075 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:01:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.861+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591749e38b0 con 0x559174a8bb80 2024-04-17T08:01:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.861+0000 7f9b11800640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174af47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.863+0000 7f9b11800640 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174af47c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27f4c0354b1b5a9a server_cookie=313a03dceabfeaad in_seq=0 out_seq=0 2024-04-17T08:01:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.863+0000 7f9af3400640 10 client.26075 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9af3400640 1 -- 172.21.15.46:0/2675641765 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591749e38b0 con 0x559174a8bb80 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9af3400640 10 client.26075 handle_command_reply: tid=0 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 client.26075 shutdown 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 msgr2=0x559174a86610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a86610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x559174a7e4a0 tx=0x7f9b04004a40 comp rx=0 tx=0).stop 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 shutdown_connections 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559174a7bb00 0x559174a86610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x559174a8bb80 0x559174af47c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559174a76990 0x559174a85600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 --2- 172.21.15.46:0/2675641765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559174a75fa0 0x559174a850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 >> 172.21.15.46:0/2675641765 conn(0x559174a71510 msgr2=0x559174a6fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.864+0000 7f9af3400640 0 client.26075 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.865+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 shutdown_connections 2024-04-17T08:01:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.866+0000 7f9b1702f740 1 -- 172.21.15.46:0/2675641765 wait complete. 2024-04-17T08:01:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.866+0000 7f9b1702f740 20 client.26075 trim_cache size 0 max 16384 2024-04-17T08:01:04.870 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:01:04.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.867+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9af40a3d00 msgr2=0x7f9af40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9af40a3d00 0x7f9af40a61c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x559174a771f0 tx=0x7f9b04009600 comp rx=0 tx=0).stop 2024-04-17T08:01:04.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1013d0 msgr2=0x7f9b0c111970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1013d0 0x7f9b0c111970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b000097d0 tx=0x7f9b00009c90 comp rx=0 tx=0).stop 2024-04-17T08:01:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 shutdown_connections 2024-04-17T08:01:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f9af40a3d00 0x7f9af40a61c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c101d90 0x7f9b0c1ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b0c1013d0 0x7f9b0c111970 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 --2- 172.21.15.46:0/698274088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b0c1001e0 0x7f9b0c111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.868+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 >> 172.21.15.46:0/698274088 conn(0x7f9b0c0fb750 msgr2=0x7f9b0c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.869+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 shutdown_connections 2024-04-17T08:01:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:04.870+0000 7f9b14200640 1 -- 172.21.15.46:0/698274088 wait complete. 2024-04-17T08:01:04.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T08:01:05.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:04 smithi062 ceph-mon[36507]: pgmap v494: 129 pgs: 129 active+clean; 50 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 518 op/s 2024-04-17T08:01:05.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:04 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2523185122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:01:05.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 -- 172.21.15.46:0/929885574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4105d60 msgr2=0x7fb1a410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/929885574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4105d60 0x7fb1a410c5f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009ca0 tx=0x7fb19c02c100 comp rx=0 tx=0).stop 2024-04-17T08:01:05.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 -- 172.21.15.46:0/929885574 shutdown_connections 2024-04-17T08:01:05.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/929885574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a4105d60 0x7fb1a410c5f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/929885574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a41053a0 0x7fb1a4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/929885574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41041b0 0x7fb1a41045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 -- 172.21.15.46:0/929885574 >> 172.21.15.46:0/929885574 conn(0x7fb1a40ff720 msgr2=0x7fb1a4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:05.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 -- 172.21.15.46:0/929885574 shutdown_connections 2024-04-17T08:01:05.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.100+0000 7fb1a9a00640 1 -- 172.21.15.46:0/929885574 wait complete. 2024-04-17T08:01:05.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 Processor -- start 2024-04-17T08:01:05.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 -- start start 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a41041b0 0x7fb1a419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41053a0 0x7fb1a419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4105d60 0x7fb1a41a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410fe40 con 0x7fb1a41053a0 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410fcc0 con 0x7fb1a41041b0 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.102+0000 7fb1a9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410ffc0 con 0x7fb1a4105d60 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41053a0 0x7fb1a419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41053a0 0x7fb1a419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45130/0 (socket says 172.21.15.46:45130) 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2000640 1 -- 172.21.15.46:0/409589967 learned_addr learned my addr 172.21.15.46:0/409589967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:05.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a41041b0 0x7fb1a419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a3400640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4105d60 0x7fb1a41a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2a00640 1 -- 172.21.15.46:0/409589967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4105d60 msgr2=0x7fb1a41a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4105d60 0x7fb1a41a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2a00640 1 -- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41053a0 msgr2=0x7fb1a419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41053a0 0x7fb1a419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2a00640 1 -- 172.21.15.46:0/409589967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41a20b0 con 0x7fb1a41041b0 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a3400640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4105d60 0x7fb1a41a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.103+0000 7fb1a2000640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41053a0 0x7fb1a419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.104+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a41041b0 0x7fb1a419a490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb19400d6a0 tx=0x7fb19400db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.104+0000 7fb19b400640 1 -- 172.21.15.46:0/409589967 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194017070 con 0x7fb1a41041b0 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.105+0000 7fb19b400640 1 -- 172.21.15.46:0/409589967 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb194009d00 con 0x7fb1a41041b0 2024-04-17T08:01:05.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.105+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41a23a0 con 0x7fb1a41041b0 2024-04-17T08:01:05.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.105+0000 7fb19b400640 1 -- 172.21.15.46:0/409589967 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194003c60 con 0x7fb1a41041b0 2024-04-17T08:01:05.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.105+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41a2940 con 0x7fb1a41041b0 2024-04-17T08:01:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.110+0000 7fb1ac79a740 1 Processor -- start 2024-04-17T08:01:05.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:04 smithi046 ceph-mon[32038]: pgmap v494: 129 pgs: 129 active+clean; 50 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 518 op/s 2024-04-17T08:01:05.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:04 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2523185122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:01:05.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.110+0000 7fb1ac79a740 1 -- start start 2024-04-17T08:01:05.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b57490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b59f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563fc8ab8020 con 0x563fc8b57b00 2024-04-17T08:01:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563fc8a5e460 con 0x563fc8b52990 2024-04-17T08:01:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563fc8ab3fe0 con 0x563fc8b51fa0 2024-04-17T08:01:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb19b400640 1 -- 172.21.15.46:0/409589967 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb194009820 con 0x7fb1a41041b0 2024-04-17T08:01:05.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1a2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b57490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b59f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b59f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45134/0 (socket says 172.21.15.46:45134) 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1a2000640 1 -- 172.21.15.46:0/3086838962 learned_addr learned my addr 172.21.15.46:0/3086838962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.111+0000 7fb1a3400640 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.112+0000 7fb1a2000640 1 -- 172.21.15.46:0/3086838962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 msgr2=0x563fc8b523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.112+0000 7fb1a2000640 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.112+0000 7fb1a2000640 1 -- 172.21.15.46:0/3086838962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 msgr2=0x563fc8b57490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.112+0000 7fb1a2000640 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b57490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.112+0000 7fb1a2000640 1 -- 172.21.15.46:0/3086838962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc8a66040 con 0x563fc8b57b00 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.112+0000 7fb1a3400640 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.113+0000 7fb1a2000640 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b59f90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1900115f0 tx=0x7fb190011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6f75d82188faa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.113+0000 7fb19b400640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3c30 0x7fb1880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.113+0000 7fb19b400640 1 -- 172.21.15.46:0/409589967 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb1940cffe0 con 0x7fb1a41041b0 2024-04-17T08:01:05.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.113+0000 7fb18fe00640 1 -- 172.21.15.46:0/3086838962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19001e070 con 0x563fc8b57b00 2024-04-17T08:01:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.113+0000 7fb18fe00640 1 -- 172.21.15.46:0/3086838962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19000fba0 con 0x563fc8b57b00 2024-04-17T08:01:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.113+0000 7fb18fe00640 1 -- 172.21.15.46:0/3086838962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190019d40 con 0x563fc8b57b00 2024-04-17T08:01:05.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.113+0000 7fb1a3400640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3c30 0x7fb1880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 -- 172.21.15.46:0/3086838962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 msgr2=0x563fc8b59f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b59f90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1900115f0 tx=0x7fb190011ab0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 -- 172.21.15.46:0/3086838962 shutdown_connections 2024-04-17T08:01:05.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b59f90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b57490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/3086838962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 -- 172.21.15.46:0/3086838962 >> 172.21.15.46:0/3086838962 conn(0x563fc8b4d510 msgr2=0x563fc8b4d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1a3400640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3c30 0x7fb1880a60f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb19c02c640 tx=0x7fb19c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 -- 172.21.15.46:0/3086838962 shutdown_connections 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.114+0000 7fb1ac79a740 1 -- 172.21.15.46:0/3086838962 wait complete. 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.115+0000 7fb1ac79a740 1 -- start start 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.115+0000 7fb1ac79a740 1 Processor -- start 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.115+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.115+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b5c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.115+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563fc8b5aab0 con 0x563fc8b57b00 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563fc8a71830 con 0x563fc8b51fa0 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563fc8b5ac30 con 0x563fc8b52990 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1ac79a740 10 client.0 _ll_register_callbacks cb 0x563fc8a3ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b5c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35732/0 (socket says 172.21.15.46:35732) 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 -- 172.21.15.46:0/4059041975 learned_addr learned my addr 172.21.15.46:0/4059041975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b5c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36686/0 (socket says 172.21.15.46:36686) 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a3400640 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b63a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 -- 172.21.15.46:0/4059041975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 msgr2=0x563fc8b5c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b5c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 -- 172.21.15.46:0/4059041975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 msgr2=0x563fc8b63a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b63a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.116+0000 7fb1a2a00640 1 -- 172.21.15.46:0/4059041975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc8a75920 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb194009610 tx=0x7fb1940042a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff66d66973a26df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb18cc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb18cc00640 1 -- 172.21.15.46:0/4059041975 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194020070 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb18cc00640 1 -- 172.21.15.46:0/4059041975 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb194004340 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19000fba0 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb18cc00640 1 -- 172.21.15.46:0/4059041975 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb19409b940 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb1ac79a740 10 client.26087 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fc8a437f0 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.117+0000 7fb18cc00640 1 -- 172.21.15.46:0/4059041975 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19000fba0 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb18cc00640 1 -- 172.21.15.46:0/4059041975 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fb1940d2110 con 0x563fc8b51fa0 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb1ac79a740 10 client.26087 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb1ac79a740 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb1ac79a740 20 client.26087 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8b67f00 0x563fc8b6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb1ac79a740 4 client.26087 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc8ac5a40 con 0x563fc8b67f00 2024-04-17T08:01:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.118+0000 7fb1a2000640 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8b67f00 0x563fc8b6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.119+0000 7fb1a2000640 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8b67f00 0x563fc8b6a3c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd639b0fb727d8ad server_cookie=62bf143e52994174 in_seq=0 out_seq=0 2024-04-17T08:01:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.119+0000 7fb18cc00640 10 client.26087 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb18cc00640 1 -- 172.21.15.46:0/4059041975 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fc8ac5a40 con 0x563fc8b67f00 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb18cc00640 10 client.26087 handle_command_reply: tid=0 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb1ac79a740 1 client.26087 shutdown 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 msgr2=0x563fc8b5c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb194009610 tx=0x7fb1940042a0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 shutdown_connections 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8b67f00 0x563fc8b6a3c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.122+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b5c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.123+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/4059041975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c4a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.123+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 >> 172.21.15.46:0/4059041975 conn(0x563fc8b4d510 msgr2=0x563fc8b4c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.123+0000 7fb18cc00640 0 client.26087 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.123+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 shutdown_connections 2024-04-17T08:01:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.123+0000 7fb1ac79a740 1 -- 172.21.15.46:0/4059041975 wait complete. 2024-04-17T08:01:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.123+0000 7fb1ac79a740 20 client.26087 trim_cache size 0 max 16384 2024-04-17T08:01:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 Processor -- start 2024-04-17T08:01:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 -- start start 2024-04-17T08:01:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41a6e80 con 0x563fc8b57b00 2024-04-17T08:01:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41a4740 con 0x563fc8b51fa0 2024-04-17T08:01:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.138+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4073b80 con 0x563fc8b52990 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b50e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45156/0 (socket says 172.21.15.46:45156) 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a3400640 1 -- 172.21.15.46:0/359871872 learned_addr learned my addr 172.21.15.46:0/359871872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2000640 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2a00640 1 -- 172.21.15.46:0/359871872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 msgr2=0x563fc8b639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2a00640 1 -- 172.21.15.46:0/359871872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 msgr2=0x563fc8b50e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b50e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2a00640 1 -- 172.21.15.46:0/359871872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc87ca3f0 con 0x563fc8b51fa0 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb194098440 tx=0x7fb1940988f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03cc92a2c85f9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb18f400640 1 -- 172.21.15.46:0/359871872 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194020070 con 0x563fc8b51fa0 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb18f400640 1 -- 172.21.15.46:0/359871872 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19409bfe0 con 0x563fc8b51fa0 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.139+0000 7fb18f400640 1 -- 172.21.15.46:0/359871872 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1940d2110 con 0x563fc8b51fa0 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 -- 172.21.15.46:0/359871872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 msgr2=0x563fc8b5c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb194098440 tx=0x7fb1940988f0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 -- 172.21.15.46:0/359871872 shutdown_connections 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b50e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b52990 0x563fc8b639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/359871872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b51fa0 0x563fc8b5c940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 -- 172.21.15.46:0/359871872 >> 172.21.15.46:0/359871872 conn(0x563fc8b4d510 msgr2=0x563fc8b50770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 -- 172.21.15.46:0/359871872 shutdown_connections 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.140+0000 7fb1ac79a740 1 -- 172.21.15.46:0/359871872 wait complete. 2024-04-17T08:01:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 -- start start 2024-04-17T08:01:05.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 Processor -- start 2024-04-17T08:01:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b61410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b61950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b62960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41a2c20 con 0x563fc8b57b00 2024-04-17T08:01:05.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41a4070 con 0x563fc8b52990 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41a32d0 con 0x563fc8b51fa0 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.141+0000 7fb1ac79a740 10 client.0 _ll_register_callbacks cb 0x7fb1940046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b61950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b61410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b61410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36704/0 (socket says 172.21.15.46:36704) 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 -- 172.21.15.46:0/2080096833 learned_addr learned my addr 172.21.15.46:0/2080096833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a3400640 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b62960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 -- 172.21.15.46:0/2080096833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 msgr2=0x563fc8b61950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b61950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 -- 172.21.15.46:0/2080096833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 msgr2=0x563fc8b62960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b62960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 -- 172.21.15.46:0/2080096833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc88947d0 con 0x563fc8b51fa0 2024-04-17T08:01:05.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb1a2a00640 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b61410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb1940989b0 tx=0x7fb1940d3860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f3bdd48699ed54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.142+0000 7fb18cc00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb18cc00640 1 -- 172.21.15.46:0/2080096833 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1940ba060 con 0x563fc8b51fa0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb18cc00640 1 -- 172.21.15.46:0/2080096833 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb194096b50 con 0x563fc8b51fa0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19409bfe0 con 0x563fc8b51fa0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb18cc00640 1 -- 172.21.15.46:0/2080096833 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb1940d7070 con 0x563fc8b51fa0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb1ac79a740 10 client.26068 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fc8bce5c0 con 0x563fc8b51fa0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb18cc00640 1 -- 172.21.15.46:0/2080096833 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19407ae50 con 0x563fc8b51fa0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb18cc00640 1 -- 172.21.15.46:0/2080096833 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fb19409b430 con 0x563fc8b51fa0 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb1ac79a740 10 client.26068 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb1ac79a740 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.143+0000 7fb1ac79a740 20 client.26068 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:05.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.144+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8bced30 0x563fc8bd1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.144+0000 7fb1ac79a740 4 client.26068 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:01:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.144+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc8abf8b0 con 0x563fc8bced30 2024-04-17T08:01:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.144+0000 7fb1a2000640 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8bced30 0x563fc8bd1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.144+0000 7fb1a2000640 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8bced30 0x563fc8bd1150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8692f8681bf1c8 server_cookie=a4c0263df32291f in_seq=0 out_seq=0 2024-04-17T08:01:05.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.144+0000 7fb18cc00640 10 client.26068 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb18cc00640 1 -- 172.21.15.46:0/2080096833 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc8abf8b0 con 0x563fc8bced30 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb18cc00640 10 client.26068 handle_command_reply: tid=0 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 client.26068 shutdown 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 msgr2=0x563fc8b61410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b61410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb1940989b0 tx=0x7fb1940d3860 comp rx=0 tx=0).stop 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 shutdown_connections 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x563fc8bced30 0x563fc8bd1150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563fc8b57b00 0x563fc8b62960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563fc8b52990 0x563fc8b61950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 --2- 172.21.15.46:0/2080096833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563fc8b51fa0 0x563fc8b61410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 >> 172.21.15.46:0/2080096833 conn(0x563fc8b4d510 msgr2=0x563fc8b51000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb18cc00640 0 client.26068 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 shutdown_connections 2024-04-17T08:01:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 1 -- 172.21.15.46:0/2080096833 wait complete. 2024-04-17T08:01:05.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.147+0000 7fb1ac79a740 20 client.26068 trim_cache size 0 max 16384 2024-04-17T08:01:05.149 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:01:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3c30 msgr2=0x7fb1880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3c30 0x7fb1880a60f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb19c02c640 tx=0x7fb19c02e040 comp rx=0 tx=0).stop 2024-04-17T08:01:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a41041b0 msgr2=0x7fb1a419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a41041b0 0x7fb1a419a490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb19400d6a0 tx=0x7fb19400db60 comp rx=0 tx=0).stop 2024-04-17T08:01:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 shutdown_connections 2024-04-17T08:01:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb1880a3c30 0x7fb1880a60f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4105d60 0x7fb1a41a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a41053a0 0x7fb1a419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 --2- 172.21.15.46:0/409589967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1a41041b0 0x7fb1a419a490 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 >> 172.21.15.46:0/409589967 conn(0x7fb1a40ff720 msgr2=0x7fb1a41086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 shutdown_connections 2024-04-17T08:01:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:05.148+0000 7fb1a9a00640 1 -- 172.21.15.46:0/409589967 wait complete. 2024-04-17T08:01:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:04 smithi053 ceph-mon[36437]: pgmap v494: 129 pgs: 129 active+clean; 50 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 518 op/s 2024-04-17T08:01:05.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:04 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2523185122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:01:06.166 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T08:01:06.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:01:06.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T08:01:06.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 -- 172.21.15.46:0/1419930340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 msgr2=0x7f8a88110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/1419930340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 0x7f8a88110790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009c10 tx=0x7f8a8002bef0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 -- 172.21.15.46:0/1419930340 shutdown_connections 2024-04-17T08:01:06.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/1419930340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 0x7f8a88110790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/1419930340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a880749c0 0x7f8a88074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/1419930340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a88075fa0 0x7f8a880743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 -- 172.21.15.46:0/1419930340 >> 172.21.15.46:0/1419930340 conn(0x7f8a880fd340 msgr2=0x7f8a880ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 -- 172.21.15.46:0/1419930340 shutdown_connections 2024-04-17T08:01:06.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.421+0000 7f8a8dc00640 1 -- 172.21.15.46:0/1419930340 wait complete. 2024-04-17T08:01:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.422+0000 7f8a8dc00640 1 Processor -- start 2024-04-17T08:01:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.422+0000 7f8a8dc00640 1 -- start start 2024-04-17T08:01:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.422+0000 7f8a8dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a880749c0 0x7f8a8819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.422+0000 7f8a8dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88075fa0 0x7f8a8819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a8dc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 0x7f8a881a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a8dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881140b0 con 0x7f8a88075fa0 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a8dc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a88113f30 con 0x7f8a8810e3a0 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a8dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a88114230 con 0x7f8a880749c0 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a880749c0 0x7f8a8819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88075fa0 0x7f8a8819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a880749c0 0x7f8a8819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36722/0 (socket says 172.21.15.46:36722) 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 -- 172.21.15.46:0/3208314133 learned_addr learned my addr 172.21.15.46:0/3208314133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a8d200640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 0x7f8a881a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 -- 172.21.15.46:0/3208314133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 msgr2=0x7f8a881a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 0x7f8a881a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 -- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88075fa0 msgr2=0x7f8a8819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88075fa0 0x7f8a8819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a87e00640 1 -- 172.21.15.46:0/3208314133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a881a6320 con 0x7f8a880749c0 2024-04-17T08:01:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.423+0000 7f8a8d200640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 0x7f8a881a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:01:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.424+0000 7f8a87e00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a880749c0 0x7f8a8819e700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800e9f0 tx=0x7f8a7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.424+0000 7f8a85600640 1 -- 172.21.15.46:0/3208314133 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800ccc0 con 0x7f8a880749c0 2024-04-17T08:01:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.424+0000 7f8a85600640 1 -- 172.21.15.46:0/3208314133 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a78005500 con 0x7f8a880749c0 2024-04-17T08:01:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.424+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a881a6610 con 0x7f8a880749c0 2024-04-17T08:01:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.424+0000 7f8a85600640 1 -- 172.21.15.46:0/3208314133 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78010690 con 0x7f8a880749c0 2024-04-17T08:01:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.424+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a881a6b10 con 0x7f8a880749c0 2024-04-17T08:01:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.427+0000 7f8a85600640 1 -- 172.21.15.46:0/3208314133 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8a780046c0 con 0x7f8a880749c0 2024-04-17T08:01:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 Processor -- start 2024-04-17T08:01:06.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 -- start start 2024-04-17T08:01:06.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cc0a0 0x56051e1cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56051e131cb0 con 0x56051e1d1c00 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56051e0d8370 con 0x56051e1cca90 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a91df3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56051e12dc70 con 0x56051e1cc0a0 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a8d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cc0a0 0x56051e1cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a8d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cc0a0 0x56051e1cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36724/0 (socket says 172.21.15.46:36724) 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.428+0000 7f8a8d200640 1 -- 172.21.15.46:0/3694980003 learned_addr learned my addr 172.21.15.46:0/3694980003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a7f400640 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a8d200640 1 -- 172.21.15.46:0/3694980003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 msgr2=0x56051e1d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a8d200640 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a8d200640 1 -- 172.21.15.46:0/3694980003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 msgr2=0x56051e1d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a8d200640 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a8d200640 1 -- 172.21.15.46:0/3694980003 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56051e0f7910 con 0x56051e1cc0a0 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a7f400640 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a85600640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8a6c0a3bb0 0x7f8a6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a85600640 1 -- 172.21.15.46:0/3208314133 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8a780d0b20 con 0x7f8a880749c0 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a87e00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8a6c0a3bb0 0x7f8a6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.429+0000 7f8a8d200640 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cc0a0 0x56051e1cc4c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8a80034fd0 tx=0x7f8a80009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf0c98814be8e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.430+0000 7f8a7d600640 1 -- 172.21.15.46:0/3694980003 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80043070 con 0x56051e1cc0a0 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.430+0000 7f8a7d600640 1 -- 172.21.15.46:0/3694980003 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a80033bc0 con 0x56051e1cc0a0 2024-04-17T08:01:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.430+0000 7f8a7d600640 1 -- 172.21.15.46:0/3694980003 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8002cb80 con 0x56051e1cc0a0 2024-04-17T08:01:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 -- 172.21.15.46:0/3694980003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cc0a0 msgr2=0x56051e1cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cc0a0 0x56051e1cc4c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8a80034fd0 tx=0x7f8a80009840 comp rx=0 tx=0).stop 2024-04-17T08:01:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a87e00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8a6c0a3bb0 0x7f8a6c0a6070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8a78098e70 tx=0x7f8a780d4560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 -- 172.21.15.46:0/3694980003 shutdown_connections 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 --2- 172.21.15.46:0/3694980003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cc0a0 0x56051e1cc4c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 -- 172.21.15.46:0/3694980003 >> 172.21.15.46:0/3694980003 conn(0x56051e1c7610 msgr2=0x56051e1c7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 -- 172.21.15.46:0/3694980003 shutdown_connections 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 -- 172.21.15.46:0/3694980003 wait complete. 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 -- start start 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.431+0000 7f8a91df3740 1 Processor -- start 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cc0a0 0x56051e1d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cca90 0x56051e1d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a91df3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56051e1d4b50 con 0x56051e1d1c00 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a91df3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56051e0eb730 con 0x56051e1cc0a0 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a91df3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56051e1d4cd0 con 0x56051e1cca90 2024-04-17T08:01:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a91df3740 10 client.0 _ll_register_callbacks cb 0x56051e0edb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a7f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cca90 0x56051e1d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a7f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cca90 0x56051e1d6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36738/0 (socket says 172.21.15.46:36738) 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a7f400640 1 -- 172.21.15.46:0/723573157 learned_addr learned my addr 172.21.15.46:0/723573157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.432+0000 7f8a8d200640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cc0a0 0x56051e1d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a8d200640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a7f400640 1 -- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cc0a0 msgr2=0x56051e1d6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a7f400640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cc0a0 0x56051e1d6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a7f400640 1 -- 172.21.15.46:0/723573157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 msgr2=0x56051e1ddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a7f400640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1ddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a7f400640 1 -- 172.21.15.46:0/723573157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56051df0c7e0 con 0x56051e1cca90 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a8d200640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1ddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a7f400640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cca90 0x56051e1d6b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a70012700 tx=0x7f8a70012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=319591a2dec33847 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a76000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a76000640 1 -- 172.21.15.46:0/723573157 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000f940 con 0x56051e1cca90 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a80033bc0 con 0x56051e1cca90 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.433+0000 7f8a76000640 1 -- 172.21.15.46:0/723573157 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8a7000fae0 con 0x56051e1cca90 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a76000640 1 -- 172.21.15.46:0/723573157 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a7000fd20 con 0x56051e1cca90 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a91df3740 10 client.26080 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56051df6cf70 con 0x56051e1cca90 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a76000640 1 -- 172.21.15.46:0/723573157 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a80033bc0 con 0x56051e1cca90 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a76000640 1 -- 172.21.15.46:0/723573157 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f8a7001c420 con 0x56051e1cca90 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a91df3740 10 client.26080 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a91df3740 10 client.26080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.434+0000 7f8a91df3740 20 client.26080 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.435+0000 7f8a91df3740 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e1e25d0 0x56051e1e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.435+0000 7f8a91df3740 4 client.26080 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.435+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56051e13f6d0 con 0x56051e1e25d0 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.435+0000 7f8a8d200640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e1e25d0 0x56051e1e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.435+0000 7f8a8d200640 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e1e25d0 0x56051e1e49c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61eedce18a16f7a server_cookie=40e8465c1f26b3c5 in_seq=0 out_seq=0 2024-04-17T08:01:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.435+0000 7f8a76000640 10 client.26080 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.436+0000 7f8a76000640 1 -- 172.21.15.46:0/723573157 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56051e13f6d0 con 0x56051e1e25d0 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.436+0000 7f8a76000640 10 client.26080 handle_command_reply: tid=0 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.436+0000 7f8a91df3740 1 client.26080 shutdown 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cca90 msgr2=0x56051e1d6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cca90 0x56051e1d6b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a70012700 tx=0x7f8a70012bc0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 shutdown_connections 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e1e25d0 0x56051e1e49c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1d1c00 0x56051e1ddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1cca90 0x56051e1d6b80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a76000640 0 client.26080 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 --2- 172.21.15.46:0/723573157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cc0a0 0x56051e1d6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 >> 172.21.15.46:0/723573157 conn(0x56051e1c7610 msgr2=0x56051e1c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 shutdown_connections 2024-04-17T08:01:06.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 1 -- 172.21.15.46:0/723573157 wait complete. 2024-04-17T08:01:06.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.437+0000 7f8a91df3740 20 client.26080 trim_cache size 0 max 16384 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 Processor -- start 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 -- start start 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 0x56051e1c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881b39d0 con 0x56051e1cc0a0 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881b1290 con 0x56051e1cca90 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.458+0000 7f8a91df3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881af0e0 con 0x56051e1d1c00 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35798/0 (socket says 172.21.15.46:35798) 2024-04-17T08:01:06.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 -- 172.21.15.46:0/1441084034 learned_addr learned my addr 172.21.15.46:0/1441084034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a8d200640 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 -- 172.21.15.46:0/1441084034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 msgr2=0x56051e1c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 0x56051e1c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 -- 172.21.15.46:0/1441084034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 msgr2=0x56051e1c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 -- 172.21.15.46:0/1441084034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56051de04170 con 0x56051e1cca90 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.459+0000 7f8a7f400640 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1ddb80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009840 tx=0x7f8a70009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf92cfb980bec9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a7cc00640 1 -- 172.21.15.46:0/1441084034 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7002f070 con 0x56051e1cca90 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a7cc00640 1 -- 172.21.15.46:0/1441084034 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7001cb50 con 0x56051e1cca90 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a7cc00640 1 -- 172.21.15.46:0/1441084034 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70005540 con 0x56051e1cca90 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a91df3740 1 -- 172.21.15.46:0/1441084034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 msgr2=0x56051e1ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1ddb80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009840 tx=0x7f8a70009d60 comp rx=0 tx=0).stop 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a91df3740 1 -- 172.21.15.46:0/1441084034 shutdown_connections 2024-04-17T08:01:06.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 0x56051e1c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1ddb80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1441084034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.460+0000 7f8a91df3740 1 -- 172.21.15.46:0/1441084034 >> 172.21.15.46:0/1441084034 conn(0x56051e1c7610 msgr2=0x56051e1d5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.461+0000 7f8a91df3740 1 -- 172.21.15.46:0/1441084034 shutdown_connections 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.461+0000 7f8a91df3740 1 -- 172.21.15.46:0/1441084034 wait complete. 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.461+0000 7f8a91df3740 1 -- start start 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.461+0000 7f8a91df3740 1 Processor -- start 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1db360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1db8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a91df3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 0x56051e1dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a91df3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881af750 con 0x56051e1cc0a0 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a91df3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881b0bc0 con 0x56051e1cca90 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a91df3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881afe20 con 0x56051e1d1c00 2024-04-17T08:01:06.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a91df3740 10 client.0 _ll_register_callbacks cb 0x7f8a70009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1db8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a8d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1db360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1db8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35812/0 (socket says 172.21.15.46:35812) 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 -- 172.21.15.46:0/1044983559 learned_addr learned my addr 172.21.15.46:0/1044983559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a8d200640 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 0x56051e1dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 -- 172.21.15.46:0/1044983559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 msgr2=0x56051e1dc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 0x56051e1dc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 -- 172.21.15.46:0/1044983559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 msgr2=0x56051e1db360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1db360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a7f400640 1 -- 172.21.15.46:0/1044983559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56051e1dcea0 con 0x56051e1cca90 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.462+0000 7f8a8d200640 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1db360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a7f400640 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1db8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009e90 tx=0x7f8a70005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62bb078c3eee7ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a76000640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a76000640 1 -- 172.21.15.46:0/1044983559 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7002f070 con 0x56051e1cca90 2024-04-17T08:01:06.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a7001cb50 con 0x56051e1cca90 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a76000640 1 -- 172.21.15.46:0/1044983559 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8a7001c630 con 0x56051e1cca90 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a76000640 1 -- 172.21.15.46:0/1044983559 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a7001ce00 con 0x56051e1cca90 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a91df3740 10 client.26111 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56051e248ba0 con 0x56051e1cca90 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a76000640 1 -- 172.21.15.46:0/1044983559 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7001cb50 con 0x56051e1cca90 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.463+0000 7f8a76000640 1 -- 172.21.15.46:0/1044983559 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f8a70003950 con 0x56051e1cca90 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a91df3740 10 client.26111 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a91df3740 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a91df3740 20 client.26111 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e249460 0x56051e24b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a91df3740 4 client.26111 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56051e139540 con 0x56051e249460 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a8d200640 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e249460 0x56051e24b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a8d200640 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e249460 0x56051e24b880 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ec70f734285ab8 server_cookie=2d070ccd1f974bbe in_seq=0 out_seq=0 2024-04-17T08:01:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.464+0000 7f8a76000640 10 client.26111 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.465+0000 7f8a76000640 1 -- 172.21.15.46:0/1044983559 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56051e139540 con 0x56051e249460 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.465+0000 7f8a76000640 10 client.26111 handle_command_reply: tid=0 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.465+0000 7f8a91df3740 1 client.26111 shutdown 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.465+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 msgr2=0x56051e1db8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.465+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1db8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009e90 tx=0x7f8a70005f10 comp rx=0 tx=0).stop 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 shutdown_connections 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56051e249460 0x56051e24b880 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56051e1d1c00 0x56051e1dc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56051e1cca90 0x56051e1db8a0 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009e90 tx=0x7f8a70005f10 comp rx=0 tx=0).stop 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 --2- 172.21.15.46:0/1044983559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56051e1cc0a0 0x56051e1db360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 >> 172.21.15.46:0/1044983559 conn(0x56051e1c7610 msgr2=0x56051e1ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a76000640 0 client.26111 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 shutdown_connections 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 1 -- 172.21.15.46:0/1044983559 wait complete. 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.466+0000 7f8a91df3740 20 client.26111 trim_cache size 0 max 16384 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "07100cf3-8ce5-4d00-9289-232d74d59ba2", 2024-04-17T08:01:06.469 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-17T08:01:06.470 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:01:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8a6c0a3bb0 msgr2=0x7f8a6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8a6c0a3bb0 0x7f8a6c0a6070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8a78098e70 tx=0x7f8a780d4560 comp rx=0 tx=0).stop 2024-04-17T08:01:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a880749c0 msgr2=0x7f8a8819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a880749c0 0x7f8a8819e700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800e9f0 tx=0x7f8a7800eeb0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 shutdown_connections 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8a6c0a3bb0 0x7f8a6c0a6070 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a8810e3a0 0x7f8a881a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88075fa0 0x7f8a8819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 --2- 172.21.15.46:0/3208314133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a880749c0 0x7f8a8819e700 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 >> 172.21.15.46:0/3208314133 conn(0x7f8a880fd340 msgr2=0x7f8a88101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 shutdown_connections 2024-04-17T08:01:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.467+0000 7f8a8dc00640 1 -- 172.21.15.46:0/3208314133 wait complete. 2024-04-17T08:01:06.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:01:06.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:01:06.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 -- 172.21.15.46:0/2251459791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c01053b0 msgr2=0x7f45c0105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 --2- 172.21.15.46:0/2251459791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c01053b0 0x7f45c0105830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009900 tx=0x7f45b402bc40 comp rx=0 tx=0).stop 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 -- 172.21.15.46:0/2251459791 shutdown_connections 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 --2- 172.21.15.46:0/2251459791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c0105d70 0x7f45c010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 --2- 172.21.15.46:0/2251459791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c01053b0 0x7f45c0105830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 --2- 172.21.15.46:0/2251459791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 0x7f45c01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 -- 172.21.15.46:0/2251459791 >> 172.21.15.46:0/2251459791 conn(0x7f45c00ff750 msgr2=0x7f45c0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 -- 172.21.15.46:0/2251459791 shutdown_connections 2024-04-17T08:01:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.753+0000 7f45c8600640 1 -- 172.21.15.46:0/2251459791 wait complete. 2024-04-17T08:01:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.754+0000 7f45c8600640 1 Processor -- start 2024-04-17T08:01:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c8600640 1 -- start start 2024-04-17T08:01:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c8600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 0x7f45c019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c01053b0 0x7f45c019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c8600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c0105d70 0x7f45c01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45c010fea0 con 0x7f45c01053b0 2024-04-17T08:01:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c8600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45c010fd20 con 0x7f45c01041c0 2024-04-17T08:01:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c8600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f45c0110020 con 0x7f45c0105d70 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 0x7f45c019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 0x7f45c019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35830/0 (socket says 172.21.15.46:35830) 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c0105d70 0x7f45c01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.755+0000 7f45c5c00640 1 -- 172.21.15.46:0/1118626819 learned_addr learned my addr 172.21.15.46:0/1118626819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.756+0000 7f45c5200640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c01053b0 0x7f45c019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.756+0000 7f45c6600640 1 -- 172.21.15.46:0/1118626819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 msgr2=0x7f45c019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.756+0000 7f45c6600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 0x7f45c019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.756+0000 7f45c6600640 1 -- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c01053b0 msgr2=0x7f45c019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.756+0000 7f45c6600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c01053b0 0x7f45c019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.756+0000 7f45c6600640 1 -- 172.21.15.46:0/1118626819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c01a20e0 con 0x7f45c0105d70 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.756+0000 7f45c5c00640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 0x7f45c019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:06.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.757+0000 7f45c6600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c0105d70 0x7f45c01a1a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009fd0 tx=0x7f45b800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.757+0000 7f45be000640 1 -- 172.21.15.46:0/1118626819 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800ce20 con 0x7f45c0105d70 2024-04-17T08:01:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.757+0000 7f45be000640 1 -- 172.21.15.46:0/1118626819 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b8005020 con 0x7f45c0105d70 2024-04-17T08:01:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.757+0000 7f45be000640 1 -- 172.21.15.46:0/1118626819 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8010720 con 0x7f45c0105d70 2024-04-17T08:01:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.757+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c01a23d0 con 0x7f45c0105d70 2024-04-17T08:01:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.757+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c01a28a0 con 0x7f45c0105d70 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.760+0000 7f45be000640 1 -- 172.21.15.46:0/1118626819 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f45b8005540 con 0x7f45c0105d70 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.760+0000 7f45cb3fb740 1 Processor -- start 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45cb3fb740 1 -- start start 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341190 0x5580c23415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2346680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c2349180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580c22a8120 con 0x5580c2341190 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5580c224e340 con 0x5580c2341b80 2024-04-17T08:01:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5580c22a40e0 con 0x5580c2346cf0 2024-04-17T08:01:06.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2346680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2346680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35834/0 (socket says 172.21.15.46:35834) 2024-04-17T08:01:06.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45c5c00640 1 -- 172.21.15.46:0/2992897654 learned_addr learned my addr 172.21.15.46:0/2992897654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.761+0000 7f45c5200640 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341190 0x5580c23415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5200640 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c2349180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5c00640 1 -- 172.21.15.46:0/2992897654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 msgr2=0x5580c2349180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5c00640 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c2349180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5c00640 1 -- 172.21.15.46:0/2992897654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341190 msgr2=0x5580c23415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5c00640 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341190 0x5580c23415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5c00640 1 -- 172.21.15.46:0/2992897654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c20847d0 con 0x5580c2341b80 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5200640 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c2349180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45be000640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45a80a3c30 0x7f45a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5c00640 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2346680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f45b00115f0 tx=0x7f45b0011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6768284630b9288c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45be000640 1 -- 172.21.15.46:0/1118626819 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f45b80d0e70 con 0x7f45c0105d70 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45c5200640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45a80a3c30 0x7f45a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45aea00640 1 -- 172.21.15.46:0/2992897654 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0004020 con 0x5580c2341b80 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45aea00640 1 -- 172.21.15.46:0/2992897654 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b00041c0 con 0x5580c2341b80 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.762+0000 7f45aea00640 1 -- 172.21.15.46:0/2992897654 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00039d0 con 0x5580c2341b80 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.763+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2992897654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 msgr2=0x5580c2346680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.763+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2346680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f45b00115f0 tx=0x7f45b0011ab0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2992897654 shutdown_connections 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c2349180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2346680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2992897654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341190 0x5580c23415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2992897654 >> 172.21.15.46:0/2992897654 conn(0x5580c233c700 msgr2=0x5580c233cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45c5200640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45a80a3c30 0x7f45a80a60f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45b402c250 tx=0x7f45b402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2992897654 shutdown_connections 2024-04-17T08:01:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2992897654 wait complete. 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.764+0000 7f45cb3fb740 1 -- start start 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 1 Processor -- start 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c234b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c234bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2346cf0 0x5580c2352c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580c2349ca0 con 0x5580c2341b80 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5580c2261710 con 0x5580c2346cf0 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5580c2349e20 con 0x5580c2341190 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45cb3fb740 10 client.0 _ll_register_callbacks cb 0x5580c225caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c234b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c234b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:36802/0 (socket says 172.21.15.46:36802) 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.765+0000 7f45c5c00640 1 -- 172.21.15.46:0/3957900982 learned_addr learned my addr 172.21.15.46:0/3957900982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5200640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c234bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c6600640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2346cf0 0x5580c2352c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5c00640 1 -- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2346cf0 msgr2=0x5580c2352c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5c00640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2346cf0 0x5580c2352c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5c00640 1 -- 172.21.15.46:0/3957900982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 msgr2=0x5580c234bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5c00640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c234bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5c00640 1 -- 172.21.15.46:0/3957900982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c1fba3f0 con 0x5580c2341190 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5200640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c234bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c6600640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2346cf0 0x5580c2352c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45c5c00640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c234b700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45b001b040 tx=0x7f45b00045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd007e9682f7743 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45a3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:06.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45a3400640 1 -- 172.21.15.46:0/3957900982 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0027020 con 0x5580c2341190 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.766+0000 7f45a3400640 1 -- 172.21.15.46:0/3957900982 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f45b0016040 con 0x5580c2341190 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.767+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b00041c0 con 0x5580c2341190 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.767+0000 7f45a3400640 1 -- 172.21.15.46:0/3957900982 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45b0032050 con 0x5580c2341190 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.767+0000 7f45cb3fb740 10 client.26092 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.767+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580c1f7a0f0 con 0x5580c2341190 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.767+0000 7f45a3400640 1 -- 172.21.15.46:0/3957900982 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b00041c0 con 0x5580c2341190 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.768+0000 7f45a3400640 1 -- 172.21.15.46:0/3957900982 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f45b00036d0 con 0x5580c2341190 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.768+0000 7f45cb3fb740 10 client.26092 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.768+0000 7f45cb3fb740 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.768+0000 7f45cb3fb740 20 client.26092 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.768+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23572b0 0x5580c2359750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.768+0000 7f45cb3fb740 4 client.26092 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.768+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c22b5b40 con 0x5580c23572b0 2024-04-17T08:01:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.769+0000 7f45c5200640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23572b0 0x5580c2359750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.769+0000 7f45c5200640 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23572b0 0x5580c2359750 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99033538df5efe1 server_cookie=70dc291748aca34e in_seq=0 out_seq=0 2024-04-17T08:01:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.769+0000 7f45a3400640 10 client.26092 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45a3400640 1 -- 172.21.15.46:0/3957900982 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580c22b5b40 con 0x5580c23572b0 2024-04-17T08:01:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45a3400640 10 client.26092 handle_command_reply: tid=0 2024-04-17T08:01:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 client.26092 shutdown 2024-04-17T08:01:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 msgr2=0x5580c234b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c234b700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45b001b040 tx=0x7f45b00045a0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 shutdown_connections 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23572b0 0x5580c2359750 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2346cf0 0x5580c2352c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c234bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/3957900982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c234b700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45a3400640 0 client.26092 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 >> 172.21.15.46:0/3957900982 conn(0x5580c233c700 msgr2=0x5580c233b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 shutdown_connections 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 1 -- 172.21.15.46:0/3957900982 wait complete. 2024-04-17T08:01:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.774+0000 7f45cb3fb740 20 client.26092 trim_cache size 0 max 16384 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 Processor -- start 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 -- start start 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c233cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2352c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2346cf0 0x5580c234ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01af650 con 0x5580c2346cf0 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01acf10 con 0x5580c2341b80 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01aad60 con 0x5580c2341190 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45c5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2352c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c233cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45c5200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2352c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35842/0 (socket says 172.21.15.46:35842) 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45c5200640 1 -- 172.21.15.46:0/829075949 learned_addr learned my addr 172.21.15.46:0/829075949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.802+0000 7f45c6600640 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2346cf0 0x5580c234ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c5c00640 1 -- 172.21.15.46:0/829075949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 msgr2=0x5580c2352c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c5c00640 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2352c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c5c00640 1 -- 172.21.15.46:0/829075949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2346cf0 msgr2=0x5580c234ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c5c00640 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2346cf0 0x5580c234ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c5c00640 1 -- 172.21.15.46:0/829075949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c2338270 con 0x5580c2341190 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c6600640 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2346cf0 0x5580c234ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c5200640 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2352c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45c5c00640 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c233cae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009600 tx=0x7f45b000fa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8892c84396eac5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45ae000640 1 -- 172.21.15.46:0/829075949 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0027040 con 0x5580c2341190 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45ae000640 1 -- 172.21.15.46:0/829075949 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b000fcf0 con 0x5580c2341190 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.803+0000 7f45ae000640 1 -- 172.21.15.46:0/829075949 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0004610 con 0x5580c2341190 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.804+0000 7f45cb3fb740 1 -- 172.21.15.46:0/829075949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 msgr2=0x5580c233cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.804+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c233cae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009600 tx=0x7f45b000fa90 comp rx=0 tx=0).stop 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 -- 172.21.15.46:0/829075949 shutdown_connections 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2346cf0 0x5580c234ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341b80 0x5580c2352c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/829075949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2341190 0x5580c233cae0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 -- 172.21.15.46:0/829075949 >> 172.21.15.46:0/829075949 conn(0x5580c233c700 msgr2=0x5580c23401f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 -- 172.21.15.46:0/829075949 shutdown_connections 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 -- 172.21.15.46:0/829075949 wait complete. 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 -- start start 2024-04-17T08:01:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.805+0000 7f45cb3fb740 1 Processor -- start 2024-04-17T08:01:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341190 0x5580c234fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c2350230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45cb3fb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c23bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01ab3d0 con 0x5580c2341b80 2024-04-17T08:01:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01ac840 con 0x5580c2341190 2024-04-17T08:01:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45cb3fb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f45c01abaa0 con 0x5580c2346cf0 2024-04-17T08:01:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45cb3fb740 10 client.0 _ll_register_callbacks cb 0x7f45b0011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341190 0x5580c234fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45c5c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341190 0x5580c234fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:35856/0 (socket says 172.21.15.46:35856) 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45c5c00640 1 -- 172.21.15.46:0/2794911254 learned_addr learned my addr 172.21.15.46:0/2794911254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45c5200640 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c2350230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.806+0000 7f45c6600640 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c23bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45c5c00640 1 -- 172.21.15.46:0/2794911254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 msgr2=0x5580c23bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45c5c00640 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c23bb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45c5c00640 1 -- 172.21.15.46:0/2794911254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 msgr2=0x5580c2350230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45c5c00640 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c2350230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45c5c00640 1 -- 172.21.15.46:0/2794911254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c23bbe40 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45c5c00640 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341190 0x5580c234fcf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f45b00058c0 tx=0x7f45b000fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca5997ee56f481 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45a3400640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45a3400640 1 -- 172.21.15.46:0/2794911254 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0027040 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.807+0000 7f45a3400640 1 -- 172.21.15.46:0/2794911254 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f45b0005260 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.808+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b000fcf0 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.808+0000 7f45a3400640 1 -- 172.21.15.46:0/2794911254 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45b00038d0 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.808+0000 7f45cb3fb740 10 client.26126 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.808+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580c23bd150 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.808+0000 7f45a3400640 1 -- 172.21.15.46:0/2794911254 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b000fcf0 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45a3400640 1 -- 172.21.15.46:0/2794911254 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f45b0023450 con 0x5580c2341190 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45cb3fb740 10 client.26126 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45cb3fb740 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45cb3fb740 20 client.26126 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23bda60 0x5580c23bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45cb3fb740 4 client.26126 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:01:06.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5580c22af9b0 con 0x5580c23bda60 2024-04-17T08:01:06.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.809+0000 7f45c5200640 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23bda60 0x5580c23bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45c5200640 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23bda60 0x5580c23bfe80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b382d14254e84a0c server_cookie=cd196101c5fd31cd in_seq=0 out_seq=0 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45a3400640 10 client.26126 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45a3400640 1 -- 172.21.15.46:0/2794911254 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5580c22af9b0 con 0x5580c23bda60 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45a3400640 10 client.26126 handle_command_reply: tid=0 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45cb3fb740 1 client.26126 shutdown 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341190 msgr2=0x5580c234fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341190 0x5580c234fcf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f45b00058c0 tx=0x7f45b000fea0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 shutdown_connections 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.810+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5580c23bda60 0x5580c23bfe80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5580c2346cf0 0x5580c23bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580c2341b80 0x5580c2350230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45cb3fb740 1 --2- 172.21.15.46:0/2794911254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580c2341190 0x5580c234fcf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 >> 172.21.15.46:0/2794911254 conn(0x5580c233c700 msgr2=0x5580c233b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45a3400640 0 client.26126 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 shutdown_connections 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45cb3fb740 1 -- 172.21.15.46:0/2794911254 wait complete. 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.811+0000 7f45cb3fb740 20 client.26126 trim_cache size 0 max 16384 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "scrub active (1001 inodes in the stack)", 2024-04-17T08:01:06.812 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-17T08:01:06.813 INFO:teuthology.orchestra.run.smithi046.stdout: "07100cf3-8ce5-4d00-9289-232d74d59ba2": { 2024-04-17T08:01:06.813 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-17T08:01:06.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "07100cf3-8ce5-4d00-9289-232d74d59ba2", 2024-04-17T08:01:06.813 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-04-17T08:01:06.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:01:06.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:01:06.813 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45a80a3c30 msgr2=0x7f45a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45a80a3c30 0x7f45a80a60f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45b402c250 tx=0x7f45b402cf90 comp rx=0 tx=0).stop 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c0105d70 msgr2=0x7f45c01a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c0105d70 0x7f45c01a1a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009fd0 tx=0x7f45b800ef10 comp rx=0 tx=0).stop 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 shutdown_connections 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f45a80a3c30 0x7f45a80a60f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45c0105d70 0x7f45c01a1a70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c01053b0 0x7f45c019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 --2- 172.21.15.46:0/1118626819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c01041c0 0x7f45c019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 >> 172.21.15.46:0/1118626819 conn(0x7f45c00ff750 msgr2=0x7f45c0108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 shutdown_connections 2024-04-17T08:01:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:06.812+0000 7f45c8600640 1 -- 172.21.15.46:0/1118626819 wait complete. 2024-04-17T08:01:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:06 smithi046 ceph-mon[32038]: pgmap v495: 129 pgs: 129 active+clean; 49 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 527 op/s 2024-04-17T08:01:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:06 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:06.832 INFO:tasks.cephfs.filesystem:scrub status for tag:07100cf3-8ce5-4d00-9289-232d74d59ba2 - {'path': '/', 'tag': '07100cf3-8ce5-4d00-9289-232d74d59ba2', 'options': 'recursive,force'} 2024-04-17T08:01:07.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:06 smithi062 ceph-mon[36507]: pgmap v495: 129 pgs: 129 active+clean; 49 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 527 op/s 2024-04-17T08:01:07.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:06 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:07.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:06 smithi053 ceph-mon[36437]: pgmap v495: 129 pgs: 129 active+clean; 49 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 527 op/s 2024-04-17T08:01:07.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:06 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:07.624 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-04-17T08:01:07.624 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:01:07.624 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T08:01:07.624 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T08:01:07.624 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T08:01:07.624 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:01:07.624 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T08:01:07.625 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-04-17T08:01:07.626 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-04-17T08:01:07.638 INFO:tasks.workunit.client.0.smithi046.stderr:+ make -j4 2024-04-17T08:01:07.667 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize -cfi 2024-04-17T08:01:08.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub queued for path: / 2024-04-17T08:01:08.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.053 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:07 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub queued for path: / 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:07 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub queued for path: / 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:07 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:08.377 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-17T08:01:08.377 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file './config.guess' 2024-04-17T08:01:08.390 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file './config.sub' 2024-04-17T08:01:08.403 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file './install-sh' 2024-04-17T08:01:08.417 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file './ltmain.sh' 2024-04-17T08:01:08.487 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-17T08:01:08.487 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-17T08:01:08.503 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-17T08:01:08.519 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-17T08:01:08.536 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-17T08:01:08.553 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-17T08:01:08.574 INFO:tasks.workunit.client.0.smithi046.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-17T08:01:08.579 INFO:tasks.workunit.client.0.smithi046.stdout:cp include/install-sh . 2024-04-17T08:01:08.653 INFO:tasks.workunit.client.0.smithi046.stdout:aclocal -I m4 2024-04-17T08:01:09.052 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:08 smithi062 ceph-mon[36507]: pgmap v496: 129 pgs: 129 active+clean; 52 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 557 op/s 2024-04-17T08:01:09.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:08 smithi046 ceph-mon[32038]: pgmap v496: 129 pgs: 129 active+clean; 52 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 557 op/s 2024-04-17T08:01:09.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:08 smithi053 ceph-mon[36437]: pgmap v496: 129 pgs: 129 active+clean; 52 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 557 op/s 2024-04-17T08:01:10.573 INFO:tasks.workunit.client.0.smithi046.stdout:autoheader 2024-04-17T08:01:11.118 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:10 smithi046 ceph-mon[32038]: pgmap v497: 129 pgs: 129 active+clean; 53 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 476 op/s 2024-04-17T08:01:11.122 INFO:tasks.workunit.client.0.smithi046.stdout:autoconf 2024-04-17T08:01:11.223 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:10 smithi053 ceph-mon[36437]: pgmap v497: 129 pgs: 129 active+clean; 53 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 476 op/s 2024-04-17T08:01:11.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:10 smithi062 ceph-mon[36507]: pgmap v497: 129 pgs: 129 active+clean; 53 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 476 op/s 2024-04-17T08:01:11.417 INFO:tasks.workunit.client.0.smithi046.stdout:./configure \ 2024-04-17T08:01:11.417 INFO:tasks.workunit.client.0.smithi046.stdout: --libexecdir=/usr/lib \ 2024-04-17T08:01:11.417 INFO:tasks.workunit.client.0.smithi046.stdout: --exec_prefix=/var/lib 2024-04-17T08:01:11.863 INFO:tasks.workunit.client.0.smithi046.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-17T08:01:11.863 INFO:tasks.workunit.client.0.smithi046.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-17T08:01:11.868 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to print strings... printf 2024-04-17T08:01:11.868 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc... gcc 2024-04-17T08:01:12.062 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether the C compiler works... yes 2024-04-17T08:01:12.063 INFO:tasks.workunit.client.0.smithi046.stdout:checking for C compiler default output file name... a.out 2024-04-17T08:01:12.153 INFO:tasks.workunit.client.0.smithi046.stdout:checking for suffix of executables... 2024-04-17T08:01:12.253 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are cross compiling... no 2024-04-17T08:01:12.346 INFO:tasks.workunit.client.0.smithi046.stdout:checking for suffix of object files... o 2024-04-17T08:01:12.416 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are using the GNU C compiler... yes 2024-04-17T08:01:12.482 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether gcc accepts -g... yes 2024-04-17T08:01:12.634 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-17T08:01:12.650 INFO:tasks.workunit.client.0.smithi046.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-17T08:01:12.664 INFO:tasks.workunit.client.0.smithi046.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-17T08:01:12.670 INFO:tasks.workunit.client.0.smithi046.stdout:checking for egrep... /usr/bin/grep -E 2024-04-17T08:01:12.675 INFO:tasks.workunit.client.0.smithi046.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-17T08:01:12.689 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-17T08:01:12.696 INFO:tasks.workunit.client.0.smithi046.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-17T08:01:12.711 INFO:tasks.workunit.client.0.smithi046.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-17T08:01:13.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:12 smithi046 ceph-mon[32038]: pgmap v498: 129 pgs: 129 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 479 op/s 2024-04-17T08:01:13.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:12 smithi053 ceph-mon[36437]: pgmap v498: 129 pgs: 129 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 479 op/s 2024-04-17T08:01:13.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:12 smithi062 ceph-mon[36507]: pgmap v498: 129 pgs: 129 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 479 op/s 2024-04-17T08:01:13.837 INFO:tasks.workunit.client.0.smithi046.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-17T08:01:13.837 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether ln -s works... yes 2024-04-17T08:01:13.884 INFO:tasks.workunit.client.0.smithi046.stdout:checking the maximum length of command line arguments... 1572864 2024-04-17T08:01:13.895 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-17T08:01:13.895 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-17T08:01:13.896 INFO:tasks.workunit.client.0.smithi046.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-17T08:01:13.896 INFO:tasks.workunit.client.0.smithi046.stdout:checking for objdump... objdump 2024-04-17T08:01:13.897 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to recognize dependent libraries... pass_all 2024-04-17T08:01:13.898 INFO:tasks.workunit.client.0.smithi046.stdout:checking for dlltool... no 2024-04-17T08:01:13.898 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-17T08:01:13.898 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ar... ar 2024-04-17T08:01:14.069 INFO:tasks.workunit.client.0.smithi046.stdout:checking for archiver @FILE support... @ 2024-04-17T08:01:14.069 INFO:tasks.workunit.client.0.smithi046.stdout:checking for strip... strip 2024-04-17T08:01:14.070 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ranlib... ranlib 2024-04-17T08:01:14.070 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gawk... gawk 2024-04-17T08:01:14.320 INFO:tasks.workunit.client.0.smithi046.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-17T08:01:14.326 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sysroot... no 2024-04-17T08:01:14.355 INFO:tasks.workunit.client.0.smithi046.stdout:checking for a working dd... /usr/bin/dd 2024-04-17T08:01:14.385 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-17T08:01:14.555 INFO:tasks.workunit.client.0.smithi046.stdout:checking for mt... no 2024-04-17T08:01:14.582 INFO:tasks.workunit.client.0.smithi046.stdout:checking if : is a manifest tool... no 2024-04-17T08:01:14.749 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to run the C preprocessor... gcc -E 2024-04-17T08:01:15.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:14 smithi046 ceph-mon[32038]: pgmap v499: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 256 op/s 2024-04-17T08:01:15.216 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ANSI C header files... yes 2024-04-17T08:01:15.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:14 smithi053 ceph-mon[36437]: pgmap v499: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 256 op/s 2024-04-17T08:01:15.294 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/types.h... yes 2024-04-17T08:01:15.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:14 smithi062 ceph-mon[36507]: pgmap v499: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 256 op/s 2024-04-17T08:01:15.387 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/stat.h... yes 2024-04-17T08:01:15.480 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdlib.h... yes 2024-04-17T08:01:15.574 INFO:tasks.workunit.client.0.smithi046.stdout:checking for string.h... yes 2024-04-17T08:01:15.678 INFO:tasks.workunit.client.0.smithi046.stdout:checking for memory.h... yes 2024-04-17T08:01:15.784 INFO:tasks.workunit.client.0.smithi046.stdout:checking for strings.h... yes 2024-04-17T08:01:15.878 INFO:tasks.workunit.client.0.smithi046.stdout:checking for inttypes.h... yes 2024-04-17T08:01:15.980 INFO:tasks.workunit.client.0.smithi046.stdout:checking for stdint.h... yes 2024-04-17T08:01:16.081 INFO:tasks.workunit.client.0.smithi046.stdout:checking for unistd.h... yes 2024-04-17T08:01:16.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:15 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:01:16.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:15 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:01:16.184 INFO:tasks.workunit.client.0.smithi046.stdout:checking for dlfcn.h... yes 2024-04-17T08:01:16.199 INFO:tasks.workunit.client.0.smithi046.stdout:checking for objdir... .libs 2024-04-17T08:01:16.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:15 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:01:16.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:15 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:01:16.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:15 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:01:16.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:15 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:01:16.473 INFO:tasks.workunit.client.0.smithi046.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-17T08:01:16.473 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-17T08:01:16.566 INFO:tasks.workunit.client.0.smithi046.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-17T08:01:16.658 INFO:tasks.workunit.client.0.smithi046.stdout:checking if gcc static flag -static works... no 2024-04-17T08:01:16.812 INFO:tasks.workunit.client.0.smithi046.stdout:checking if gcc supports -c -o file.o... yes 2024-04-17T08:01:16.812 INFO:tasks.workunit.client.0.smithi046.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-17T08:01:16.840 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-17T08:01:16.940 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether -lc should be explicitly linked in... no 2024-04-17T08:01:17.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:16 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:01:17.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:16 smithi046 ceph-mon[32038]: pgmap v500: 129 pgs: 129 active+clean; 53 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 503 KiB/s wr, 143 op/s 2024-04-17T08:01:17.114 INFO:tasks.workunit.client.0.smithi046.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-17T08:01:17.115 INFO:tasks.workunit.client.0.smithi046.stdout:checking how to hardcode library paths into programs... immediate 2024-04-17T08:01:17.120 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether stripping libraries is possible... yes 2024-04-17T08:01:17.120 INFO:tasks.workunit.client.0.smithi046.stdout:checking if libtool supports shared libraries... yes 2024-04-17T08:01:17.120 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether to build shared libraries... yes 2024-04-17T08:01:17.120 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether to build static libraries... yes 2024-04-17T08:01:17.144 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc... (cached) gcc 2024-04-17T08:01:17.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:16 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:01:17.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:16 smithi053 ceph-mon[36437]: pgmap v500: 129 pgs: 129 active+clean; 53 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 503 KiB/s wr, 143 op/s 2024-04-17T08:01:17.234 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-17T08:01:17.234 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-17T08:01:17.236 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-17T08:01:17.237 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gmake... /usr/bin/gmake 2024-04-17T08:01:17.238 INFO:tasks.workunit.client.0.smithi046.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-04-17T08:01:17.239 INFO:tasks.workunit.client.0.smithi046.stdout:checking for tar... /bin/tar 2024-04-17T08:01:17.239 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gzip... /bin/gzip 2024-04-17T08:01:17.291 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether gcc -MM is supported... yes 2024-04-17T08:01:17.292 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sort... /bin/sort 2024-04-17T08:01:17.292 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether ln -s works... yes 2024-04-17T08:01:17.294 INFO:tasks.workunit.client.0.smithi046.stdout:checking for rpm... /bin/rpm 2024-04-17T08:01:17.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:16 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:01:17.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:16 smithi062 ceph-mon[36507]: pgmap v500: 129 pgs: 129 active+clean; 53 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 503 KiB/s wr, 143 op/s 2024-04-17T08:01:17.307 INFO:tasks.workunit.client.0.smithi046.stdout:checking for rpmbuild... no 2024-04-17T08:01:17.307 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ANSI C header files... (cached) yes 2024-04-17T08:01:17.429 INFO:tasks.workunit.client.0.smithi046.stdout:checking assert.h usability... yes 2024-04-17T08:01:17.473 INFO:tasks.workunit.client.0.smithi046.stdout:checking assert.h presence... yes 2024-04-17T08:01:17.473 INFO:tasks.workunit.client.0.smithi046.stdout:checking for assert.h... yes 2024-04-17T08:01:17.586 INFO:tasks.workunit.client.0.smithi046.stdout:checking bstring.h usability... no 2024-04-17T08:01:17.644 INFO:tasks.workunit.client.0.smithi046.stdout:checking bstring.h presence... no 2024-04-17T08:01:17.644 INFO:tasks.workunit.client.0.smithi046.stdout:checking for bstring.h... no 2024-04-17T08:01:17.748 INFO:tasks.workunit.client.0.smithi046.stdout:checking libgen.h usability... yes 2024-04-17T08:01:17.790 INFO:tasks.workunit.client.0.smithi046.stdout:checking libgen.h presence... yes 2024-04-17T08:01:17.790 INFO:tasks.workunit.client.0.smithi046.stdout:checking for libgen.h... yes 2024-04-17T08:01:17.913 INFO:tasks.workunit.client.0.smithi046.stdout:checking dirent.h usability... yes 2024-04-17T08:01:17.958 INFO:tasks.workunit.client.0.smithi046.stdout:checking dirent.h presence... yes 2024-04-17T08:01:17.958 INFO:tasks.workunit.client.0.smithi046.stdout:checking for dirent.h... yes 2024-04-17T08:01:18.069 INFO:tasks.workunit.client.0.smithi046.stdout:checking errno.h usability... yes 2024-04-17T08:01:18.111 INFO:tasks.workunit.client.0.smithi046.stdout:checking errno.h presence... yes 2024-04-17T08:01:18.111 INFO:tasks.workunit.client.0.smithi046.stdout:checking for errno.h... yes 2024-04-17T08:01:18.236 INFO:tasks.workunit.client.0.smithi046.stdout:checking malloc.h usability... yes 2024-04-17T08:01:18.280 INFO:tasks.workunit.client.0.smithi046.stdout:checking malloc.h presence... yes 2024-04-17T08:01:18.280 INFO:tasks.workunit.client.0.smithi046.stdout:checking for malloc.h... yes 2024-04-17T08:01:18.397 INFO:tasks.workunit.client.0.smithi046.stdout:checking uuid.h usability... no 2024-04-17T08:01:18.459 INFO:tasks.workunit.client.0.smithi046.stdout:checking uuid.h presence... no 2024-04-17T08:01:18.459 INFO:tasks.workunit.client.0.smithi046.stdout:checking for uuid.h... no 2024-04-17T08:01:18.684 INFO:tasks.workunit.client.0.smithi046.stdout:checking uuid/uuid.h usability... yes 2024-04-17T08:01:18.860 INFO:tasks.workunit.client.0.smithi046.stdout:checking uuid/uuid.h presence... yes 2024-04-17T08:01:18.861 INFO:tasks.workunit.client.0.smithi046.stdout:checking for uuid/uuid.h... yes 2024-04-17T08:01:19.068 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/uuid.h usability... no 2024-04-17T08:01:19.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:18 smithi046 ceph-mon[32038]: pgmap v501: 129 pgs: 129 active+clean; 54 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 639 KiB/s wr, 131 op/s 2024-04-17T08:01:19.127 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/uuid.h presence... no 2024-04-17T08:01:19.128 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/uuid.h... no 2024-04-17T08:01:19.219 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/file.h usability... yes 2024-04-17T08:01:19.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:18 smithi053 ceph-mon[36437]: pgmap v501: 129 pgs: 129 active+clean; 54 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 639 KiB/s wr, 131 op/s 2024-04-17T08:01:19.259 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/file.h presence... yes 2024-04-17T08:01:19.259 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/file.h... yes 2024-04-17T08:01:19.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:18 smithi062 ceph-mon[36507]: pgmap v501: 129 pgs: 129 active+clean; 54 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 639 KiB/s wr, 131 op/s 2024-04-17T08:01:19.373 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/fcntl.h usability... yes 2024-04-17T08:01:19.416 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/fcntl.h presence... yes 2024-04-17T08:01:19.416 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/fcntl.h... yes 2024-04-17T08:01:19.528 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/syssgi.h usability... no 2024-04-17T08:01:19.581 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/syssgi.h presence... no 2024-04-17T08:01:19.582 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/syssgi.h... no 2024-04-17T08:01:19.856 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/param.h usability... yes 2024-04-17T08:01:19.907 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/param.h presence... yes 2024-04-17T08:01:19.907 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/param.h... yes 2024-04-17T08:01:19.940 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/stat.h... (cached) yes 2024-04-17T08:01:20.065 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/statvfs.h usability... yes 2024-04-17T08:01:20.110 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/statvfs.h presence... yes 2024-04-17T08:01:20.110 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/statvfs.h... yes 2024-04-17T08:01:20.215 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/time.h usability... yes 2024-04-17T08:01:20.263 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/time.h presence... yes 2024-04-17T08:01:20.264 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/time.h... yes 2024-04-17T08:01:20.380 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/ioctl.h usability... yes 2024-04-17T08:01:20.426 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/ioctl.h presence... yes 2024-04-17T08:01:20.427 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/ioctl.h... yes 2024-04-17T08:01:20.559 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/wait.h usability... yes 2024-04-17T08:01:20.610 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/wait.h presence... yes 2024-04-17T08:01:20.610 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/wait.h... yes 2024-04-17T08:01:20.645 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/types.h... (cached) yes 2024-04-17T08:01:20.663 INFO:tasks.workunit.client.0.smithi046.stdout:checking for strings.h... (cached) yes 2024-04-17T08:01:20.764 INFO:tasks.workunit.client.0.smithi046.stdout:checking err.h usability... yes 2024-04-17T08:01:20.810 INFO:tasks.workunit.client.0.smithi046.stdout:checking err.h presence... yes 2024-04-17T08:01:20.811 INFO:tasks.workunit.client.0.smithi046.stdout:checking for err.h... yes 2024-04-17T08:01:20.922 INFO:tasks.workunit.client.0.smithi046.stdout:checking linux/falloc.h usability... yes 2024-04-17T08:01:20.966 INFO:tasks.workunit.client.0.smithi046.stdout:checking linux/falloc.h presence... yes 2024-04-17T08:01:20.966 INFO:tasks.workunit.client.0.smithi046.stdout:checking for linux/falloc.h... yes 2024-04-17T08:01:21.087 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-17T08:01:21.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:20 smithi046 ceph-mon[32038]: pgmap v502: 129 pgs: 129 active+clean; 55 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 392 KiB/s wr, 81 op/s 2024-04-17T08:01:21.157 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-17T08:01:21.157 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-17T08:01:21.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:20 smithi053 ceph-mon[36437]: pgmap v502: 129 pgs: 129 active+clean; 55 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 392 KiB/s wr, 81 op/s 2024-04-17T08:01:21.254 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-17T08:01:21.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:20 smithi062 ceph-mon[36507]: pgmap v502: 129 pgs: 129 active+clean; 55 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 392 KiB/s wr, 81 op/s 2024-04-17T08:01:21.319 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-17T08:01:21.319 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-17T08:01:21.419 INFO:tasks.workunit.client.0.smithi046.stdout:checking xfs/platform_defs.h usability... no 2024-04-17T08:01:21.480 INFO:tasks.workunit.client.0.smithi046.stdout:checking xfs/platform_defs.h presence... no 2024-04-17T08:01:21.480 INFO:tasks.workunit.client.0.smithi046.stdout:checking for xfs/platform_defs.h... no 2024-04-17T08:01:21.581 INFO:tasks.workunit.client.0.smithi046.stdout:checking btrfs/ioctl.h usability... no 2024-04-17T08:01:21.644 INFO:tasks.workunit.client.0.smithi046.stdout:checking btrfs/ioctl.h presence... no 2024-04-17T08:01:21.644 INFO:tasks.workunit.client.0.smithi046.stdout:checking for btrfs/ioctl.h... no 2024-04-17T08:01:21.738 INFO:tasks.workunit.client.0.smithi046.stdout:checking cifs/ioctl.h usability... no 2024-04-17T08:01:21.826 INFO:tasks.workunit.client.0.smithi046.stdout:checking cifs/ioctl.h presence... no 2024-04-17T08:01:21.826 INFO:tasks.workunit.client.0.smithi046.stdout:checking for cifs/ioctl.h... no 2024-04-17T08:01:21.908 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/mman.h usability... yes 2024-04-17T08:01:21.945 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/mman.h presence... yes 2024-04-17T08:01:21.945 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/mman.h... yes 2024-04-17T08:01:22.035 INFO:tasks.workunit.client.0.smithi046.stdout:checking for xfs/xfs_log_format.h... no 2024-04-17T08:01:22.171 INFO:tasks.workunit.client.0.smithi046.stdout:checking for uuid_compare... no 2024-04-17T08:01:22.267 INFO:tasks.workunit.client.0.smithi046.stdout:checking for uuid_compare in -luuid... yes 2024-04-17T08:01:22.420 INFO:tasks.workunit.client.0.smithi046.stdout:checking for xfs/xfs.h... yes 2024-04-17T08:01:22.498 INFO:tasks.workunit.client.0.smithi046.stdout:checking for xfs/libxfs.h... no 2024-04-17T08:01:22.582 INFO:tasks.workunit.client.0.smithi046.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-17T08:01:22.681 INFO:tasks.workunit.client.0.smithi046.stdout:checking for xfs/xqm.h... yes 2024-04-17T08:01:22.802 INFO:tasks.workunit.client.0.smithi046.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-17T08:01:22.878 INFO:tasks.workunit.client.0.smithi046.stdout:checking xfs/handle.h usability... yes 2024-04-17T08:01:22.914 INFO:tasks.workunit.client.0.smithi046.stdout:checking xfs/handle.h presence... yes 2024-04-17T08:01:22.914 INFO:tasks.workunit.client.0.smithi046.stdout:checking for xfs/handle.h... yes 2024-04-17T08:01:23.014 INFO:tasks.workunit.client.0.smithi046.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-17T08:01:23.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:22 smithi046 ceph-mon[32038]: pgmap v503: 129 pgs: 129 active+clean; 57 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 527 KiB/s wr, 136 op/s 2024-04-17T08:01:23.147 INFO:tasks.workunit.client.0.smithi046.stdout:checking attr/attributes.h usability... yes 2024-04-17T08:01:23.189 INFO:tasks.workunit.client.0.smithi046.stdout:checking attr/attributes.h presence... yes 2024-04-17T08:01:23.189 INFO:tasks.workunit.client.0.smithi046.stdout:checking for attr/attributes.h... yes 2024-04-17T08:01:23.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:22 smithi053 ceph-mon[36437]: pgmap v503: 129 pgs: 129 active+clean; 57 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 527 KiB/s wr, 136 op/s 2024-04-17T08:01:23.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:22 smithi062 ceph-mon[36507]: pgmap v503: 129 pgs: 129 active+clean; 57 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 527 KiB/s wr, 136 op/s 2024-04-17T08:01:23.304 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/attributes.h usability... no 2024-04-17T08:01:23.369 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/attributes.h presence... no 2024-04-17T08:01:23.369 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/attributes.h... no 2024-04-17T08:01:23.474 INFO:tasks.workunit.client.0.smithi046.stdout:checking for attr_list in -lattr... yes 2024-04-17T08:01:23.575 INFO:tasks.workunit.client.0.smithi046.stdout:checking for attr_set in -lattr... yes 2024-04-17T08:01:23.902 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/acl.h usability... yes 2024-04-17T08:01:23.950 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/acl.h presence... yes 2024-04-17T08:01:23.950 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/acl.h... yes 2024-04-17T08:01:24.034 INFO:tasks.workunit.client.0.smithi046.stdout:checking acl/libacl.h usability... yes 2024-04-17T08:01:24.066 INFO:tasks.workunit.client.0.smithi046.stdout:checking acl/libacl.h presence... yes 2024-04-17T08:01:24.066 INFO:tasks.workunit.client.0.smithi046.stdout:checking for acl/libacl.h... yes 2024-04-17T08:01:24.149 INFO:tasks.workunit.client.0.smithi046.stdout:checking for acl_init in -lacl... yes 2024-04-17T08:01:24.290 INFO:tasks.workunit.client.0.smithi046.stdout:checking gdbm-ndbm.h usability... no 2024-04-17T08:01:24.355 INFO:tasks.workunit.client.0.smithi046.stdout:checking gdbm-ndbm.h presence... no 2024-04-17T08:01:24.355 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gdbm-ndbm.h... no 2024-04-17T08:01:24.443 INFO:tasks.workunit.client.0.smithi046.stdout:checking gdbm/ndbm.h usability... no 2024-04-17T08:01:24.510 INFO:tasks.workunit.client.0.smithi046.stdout:checking gdbm/ndbm.h presence... no 2024-04-17T08:01:24.511 INFO:tasks.workunit.client.0.smithi046.stdout:checking for gdbm/ndbm.h... no 2024-04-17T08:01:24.604 INFO:tasks.workunit.client.0.smithi046.stdout:checking ndbm.h usability... no 2024-04-17T08:01:24.675 INFO:tasks.workunit.client.0.smithi046.stdout:checking ndbm.h presence... no 2024-04-17T08:01:24.675 INFO:tasks.workunit.client.0.smithi046.stdout:checking for ndbm.h... no 2024-04-17T08:01:24.761 INFO:tasks.workunit.client.0.smithi046.stdout:checking libaio.h usability... yes 2024-04-17T08:01:24.824 INFO:tasks.workunit.client.0.smithi046.stdout:checking libaio.h presence... yes 2024-04-17T08:01:24.824 INFO:tasks.workunit.client.0.smithi046.stdout:checking for libaio.h... yes 2024-04-17T08:01:24.956 INFO:tasks.workunit.client.0.smithi046.stdout:checking for dm_make_handle in -ldm... no 2024-04-17T08:01:24.956 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-04-17T08:01:24.956 INFO:tasks.workunit.client.0.smithi046.stdout:WARNING: could not find a valid DMAPI base library. 2024-04-17T08:01:24.956 INFO:tasks.workunit.client.0.smithi046.stdout:If you want DMAPI support please install the data migration 2024-04-17T08:01:24.957 INFO:tasks.workunit.client.0.smithi046.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-04-17T08:01:24.957 INFO:tasks.workunit.client.0.smithi046.stdout:from the dmapi source. 2024-04-17T08:01:24.957 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-04-17T08:01:25.021 INFO:tasks.workunit.client.0.smithi046.stdout:checking linux/fiemap.h usability... yes 2024-04-17T08:01:25.071 INFO:tasks.workunit.client.0.smithi046.stdout:checking linux/fiemap.h presence... yes 2024-04-17T08:01:25.071 INFO:tasks.workunit.client.0.smithi046.stdout:checking for linux/fiemap.h... yes 2024-04-17T08:01:25.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:24 smithi046 ceph-mon[32038]: pgmap v504: 129 pgs: 129 active+clean; 61 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 831 KiB/s wr, 142 op/s 2024-04-17T08:01:25.180 INFO:tasks.workunit.client.0.smithi046.stdout:checking for fallocate... yes 2024-04-17T08:01:25.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:24 smithi053 ceph-mon[36437]: pgmap v504: 129 pgs: 129 active+clean; 61 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 831 KiB/s wr, 142 op/s 2024-04-17T08:01:25.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:24 smithi062 ceph-mon[36507]: pgmap v504: 129 pgs: 129 active+clean; 61 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 831 KiB/s wr, 142 op/s 2024-04-17T08:01:25.321 INFO:tasks.workunit.client.0.smithi046.stdout:checking for open_by_handle_at... yes 2024-04-17T08:01:25.408 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/prctl.h usability... yes 2024-04-17T08:01:25.451 INFO:tasks.workunit.client.0.smithi046.stdout:checking sys/prctl.h presence... yes 2024-04-17T08:01:25.451 INFO:tasks.workunit.client.0.smithi046.stdout:checking for sys/prctl.h... yes 2024-04-17T08:01:25.532 INFO:tasks.workunit.client.0.smithi046.stdout:checking linux/fs.h usability... yes 2024-04-17T08:01:25.574 INFO:tasks.workunit.client.0.smithi046.stdout:checking linux/fs.h presence... yes 2024-04-17T08:01:25.574 INFO:tasks.workunit.client.0.smithi046.stdout:checking for linux/fs.h... yes 2024-04-17T08:01:25.674 INFO:tasks.workunit.client.0.smithi046.stdout:checking btrfsutil.h usability... no 2024-04-17T08:01:25.738 INFO:tasks.workunit.client.0.smithi046.stdout:checking btrfsutil.h presence... no 2024-04-17T08:01:25.738 INFO:tasks.workunit.client.0.smithi046.stdout:checking for btrfsutil.h... no 2024-04-17T08:01:25.864 INFO:tasks.workunit.client.0.smithi046.stdout:checking for copy_file_range... yes 2024-04-17T08:01:25.968 INFO:tasks.workunit.client.0.smithi046.stdout:checking for renameat2... yes 2024-04-17T08:01:26.014 INFO:tasks.workunit.client.0.smithi046.stdout:configure: creating ./config.status 2024-04-17T08:01:27.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:26 smithi046 ceph-mon[32038]: pgmap v505: 129 pgs: 129 active+clean; 62 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 858 KiB/s wr, 139 op/s 2024-04-17T08:01:27.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:26 smithi053 ceph-mon[36437]: pgmap v505: 129 pgs: 129 active+clean; 62 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 858 KiB/s wr, 139 op/s 2024-04-17T08:01:27.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:26 smithi062 ceph-mon[36507]: pgmap v505: 129 pgs: 129 active+clean; 62 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 858 KiB/s wr, 139 op/s 2024-04-17T08:01:27.336 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: creating include/builddefs 2024-04-17T08:01:28.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:27 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:01:28.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:27 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:01:28.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:27 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:01:29.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:28 smithi046 ceph-mon[32038]: pgmap v506: 129 pgs: 129 active+clean; 68 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-17T08:01:29.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:01:29.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:01:29.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:28 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:01:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:28 smithi053 ceph-mon[36437]: pgmap v506: 129 pgs: 129 active+clean; 68 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-17T08:01:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:01:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:01:29.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:28 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:01:29.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:28 smithi062 ceph-mon[36507]: pgmap v506: 129 pgs: 129 active+clean; 68 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-17T08:01:29.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:01:29.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:01:29.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:28 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:01:30.076 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: creating include/config.h 2024-04-17T08:01:31.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:30 smithi053 ceph-mon[36437]: pgmap v507: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-04-17T08:01:31.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:30 smithi062 ceph-mon[36507]: pgmap v507: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-04-17T08:01:31.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:30 smithi046 ceph-mon[32038]: pgmap v507: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-04-17T08:01:33.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:32 smithi053 ceph-mon[36437]: pgmap v508: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-04-17T08:01:33.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:32 smithi062 ceph-mon[36507]: pgmap v508: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-04-17T08:01:33.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:32 smithi046 ceph-mon[32038]: pgmap v508: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-04-17T08:01:35.070 INFO:tasks.workunit.client.0.smithi046.stdout:config.status: executing libtool commands 2024-04-17T08:01:35.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:34 smithi053 ceph-mon[36437]: pgmap v509: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-04-17T08:01:35.254 INFO:tasks.workunit.client.0.smithi046.stdout:Building include 2024-04-17T08:01:35.255 INFO:tasks.workunit.client.0.smithi046.stdout:Building lib 2024-04-17T08:01:35.256 INFO:tasks.workunit.client.0.smithi046.stdout:Building tests 2024-04-17T08:01:35.268 INFO:tasks.workunit.client.0.smithi046.stdout: [LTDEP] 2024-04-17T08:01:35.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:34 smithi062 ceph-mon[36507]: pgmap v509: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-04-17T08:01:35.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:34 smithi046 ceph-mon[32038]: pgmap v509: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-04-17T08:01:35.705 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] dataascii.lo 2024-04-17T08:01:35.706 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] databin.lo 2024-04-17T08:01:35.706 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] datapid.lo 2024-04-17T08:01:35.707 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] file_lock.lo 2024-04-17T08:01:36.084 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] forker.lo 2024-04-17T08:01:36.085 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] pattern.lo 2024-04-17T08:01:36.173 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] open_flags.lo 2024-04-17T08:01:36.191 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] random_range.lo 2024-04-17T08:01:36.356 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] string_to_tokens.lo 2024-04-17T08:01:36.372 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] str_to_bytes.lo 2024-04-17T08:01:36.490 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] tlibio.lo 2024-04-17T08:01:36.615 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] write_log.lo 2024-04-17T08:01:36.616 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] random.lo 2024-04-17T08:01:36.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:01:36.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:01:37.017 INFO:tasks.workunit.client.0.smithi046.stdout: [LD] 2024-04-17T08:01:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 -- 172.21.15.46:0/3449237369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec078500 msgr2=0x7f4fec078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 --2- 172.21.15.46:0/3449237369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec078500 0x7f4fec078da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009be0 tx=0x7f4fe402bef0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 -- 172.21.15.46:0/3449237369 shutdown_connections 2024-04-17T08:01:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 --2- 172.21.15.46:0/3449237369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec078500 0x7f4fec078da0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 --2- 172.21.15.46:0/3449237369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec077b60 0x7f4fec077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 --2- 172.21.15.46:0/3449237369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fec076920 0x7f4fec076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 -- 172.21.15.46:0/3449237369 >> 172.21.15.46:0/3449237369 conn(0x7f4fec0ff570 msgr2=0x7f4fec101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 -- 172.21.15.46:0/3449237369 shutdown_connections 2024-04-17T08:01:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.035+0000 7f4ff2000640 1 -- 172.21.15.46:0/3449237369 wait complete. 2024-04-17T08:01:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.036+0000 7f4ff2000640 1 Processor -- start 2024-04-17T08:01:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff2000640 1 -- start start 2024-04-17T08:01:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec076920 0x7f4fec19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec077b60 0x7f4fec19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fec078500 0x7f4fec1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1140b0 con 0x7f4fec078500 2024-04-17T08:01:37.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff2000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec113f30 con 0x7f4fec077b60 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec114230 con 0x7f4fec076920 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec076920 0x7f4fec19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fec078500 0x7f4fec1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec076920 0x7f4fec19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44002/0 (socket says 172.21.15.46:44002) 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4ff0c00640 1 -- 172.21.15.46:0/1141555654 learned_addr learned my addr 172.21.15.46:0/1141555654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.037+0000 7f4febe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec077b60 0x7f4fec19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff0c00640 1 -- 172.21.15.46:0/1141555654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec077b60 msgr2=0x7f4fec19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff0c00640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec077b60 0x7f4fec19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff0c00640 1 -- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fec078500 msgr2=0x7f4fec1a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff0c00640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fec078500 0x7f4fec1a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff0c00640 1 -- 172.21.15.46:0/1141555654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec1a62d0 con 0x7f4fec076920 2024-04-17T08:01:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff1600640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fec078500 0x7f4fec1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff0c00640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec076920 0x7f4fec19e6b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9c0 tx=0x7f4fe000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4fe9600640 1 -- 172.21.15.46:0/1141555654 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000cde0 con 0x7f4fec076920 2024-04-17T08:01:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec1a65c0 con 0x7f4fec076920 2024-04-17T08:01:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4fe9600640 1 -- 172.21.15.46:0/1141555654 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe00035f0 con 0x7f4fec076920 2024-04-17T08:01:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4fe9600640 1 -- 172.21.15.46:0/1141555654 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0010650 con 0x7f4fec076920 2024-04-17T08:01:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.038+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a6b10 con 0x7f4fec076920 2024-04-17T08:01:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 Processor -- start 2024-04-17T08:01:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 -- start start 2024-04-17T08:01:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feab370 0x55ea3feab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3feb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ea3fe12120 con 0x55ea3feabd60 2024-04-17T08:01:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ea3fdb8340 con 0x55ea3feb0ed0 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4ff6372740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ea3fe0e0e0 con 0x55ea3feab370 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.043+0000 7f4fe9600640 1 -- 172.21.15.46:0/1141555654 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4fe00037a0 con 0x7f4fec076920 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feab370 0x55ea3feab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4febe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feab370 0x55ea3feab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44004/0 (socket says 172.21.15.46:44004) 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 -- 172.21.15.46:0/1902046883 learned_addr learned my addr 172.21.15.46:0/1902046883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff0c00640 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3feb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 -- 172.21.15.46:0/1902046883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 msgr2=0x55ea3feb3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 -- 172.21.15.46:0/1902046883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 msgr2=0x55ea3feb0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3feb0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.044+0000 7f4ff1600640 1 -- 172.21.15.46:0/1902046883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3fbee7d0 con 0x55ea3feab370 2024-04-17T08:01:37.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.045+0000 7f4ff1600640 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feab370 0x55ea3feab790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402c500 tx=0x7f4fe4033ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc299d28ef77e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.045+0000 7f4fddc00640 1 -- 172.21.15.46:0/1902046883 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4043030 con 0x55ea3feab370 2024-04-17T08:01:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.045+0000 7f4fddc00640 1 -- 172.21.15.46:0/1902046883 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4003970 con 0x55ea3feab370 2024-04-17T08:01:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.045+0000 7f4fddc00640 1 -- 172.21.15.46:0/1902046883 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe402cd10 con 0x55ea3feab370 2024-04-17T08:01:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.045+0000 7f4fe9600640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4fd40a3bb0 0x7f4fd40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.045+0000 7f4fe9600640 1 -- 172.21.15.46:0/1141555654 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4fe00d0ae0 con 0x7f4fec076920 2024-04-17T08:01:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff0c00640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4fd40a3bb0 0x7f4fd40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 -- 172.21.15.46:0/1902046883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feab370 msgr2=0x55ea3feab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feab370 0x55ea3feab790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402c500 tx=0x7f4fe4033ef0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 -- 172.21.15.46:0/1902046883 shutdown_connections 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3feb0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 --2- 172.21.15.46:0/1902046883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feab370 0x55ea3feab790 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 -- 172.21.15.46:0/1902046883 >> 172.21.15.46:0/1902046883 conn(0x55ea3fea68e0 msgr2=0x55ea3fea6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 -- 172.21.15.46:0/1902046883 shutdown_connections 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff6372740 1 -- 172.21.15.46:0/1902046883 wait complete. 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.046+0000 7f4ff0c00640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4fd40a3bb0 0x7f4fd40a6070 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00993a0 tx=0x7f4fe0005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 -- start start 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 Processor -- start 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feab370 0x55ea3feb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3feb5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3febcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ea3feb3e20 con 0x55ea3feab370 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ea3fdcb710 con 0x55ea3feb0ed0 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ea3feb3fa0 con 0x55ea3feabd60 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff6372740 10 client.0 _ll_register_callbacks cb 0x55ea3fdc6af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feab370 0x55ea3feb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feab370 0x55ea3feb5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51780/0 (socket says 172.21.15.46:51780) 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.047+0000 7f4ff1600640 1 -- 172.21.15.46:0/747459653 learned_addr learned my addr 172.21.15.46:0/747459653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4febe00640 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3feb5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff1600640 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3febcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff1600640 1 -- 172.21.15.46:0/747459653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 msgr2=0x55ea3feb5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff1600640 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3feb5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff1600640 1 -- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 msgr2=0x55ea3febcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff1600640 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3febcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff1600640 1 -- 172.21.15.46:0/747459653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3fb243f0 con 0x55ea3feab370 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff1600640 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feab370 0x55ea3feb5800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402c430 tx=0x7f4fe402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fb291cdea673e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4fd2000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4fd2000640 1 -- 172.21.15.46:0/747459653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4043070 con 0x55ea3feab370 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4fd2000640 1 -- 172.21.15.46:0/747459653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4fe40348a0 con 0x55ea3feab370 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.048+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe4003970 con 0x55ea3feab370 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4fd2000640 1 -- 172.21.15.46:0/747459653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fe405b050 con 0x55ea3feab370 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4ff6372740 10 client.15978 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea3fae40f0 con 0x55ea3feab370 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4fd2000640 1 -- 172.21.15.46:0/747459653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4003970 con 0x55ea3feab370 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4fd2000640 1 -- 172.21.15.46:0/747459653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f4fe403fbf0 con 0x55ea3feab370 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4ff6372740 10 client.15978 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4ff6372740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4ff6372740 20 client.15978 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.049+0000 7f4ff6372740 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3fec1270 0x55ea3fec3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.050+0000 7f4ff6372740 4 client.15978 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.050+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea3fe1fb40 con 0x55ea3fec1270 2024-04-17T08:01:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.050+0000 7f4febe00640 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3fec1270 0x55ea3fec3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.050+0000 7f4febe00640 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3fec1270 0x55ea3fec3660 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506501446cd9c45d server_cookie=4c2901a6ffb7db2 in_seq=0 out_seq=0 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.050+0000 7f4fd2000640 10 client.15978 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.051+0000 7f4fd2000640 1 -- 172.21.15.46:0/747459653 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea3fe1fb40 con 0x55ea3fec1270 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.051+0000 7f4fd2000640 10 client.15978 handle_command_reply: tid=0 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.051+0000 7f4ff6372740 1 client.15978 shutdown 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.051+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feab370 msgr2=0x55ea3feb5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.051+0000 7f4ff6372740 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feab370 0x55ea3feb5800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402c430 tx=0x7f4fe402bef0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 shutdown_connections 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3fec1270 0x55ea3fec3660 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feb0ed0 0x55ea3febcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3feb5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 --2- 172.21.15.46:0/747459653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feab370 0x55ea3feb5800 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4fd2000640 0 client.15978 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:37.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 >> 172.21.15.46:0/747459653 conn(0x55ea3fea68e0 msgr2=0x55ea3fea5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 shutdown_connections 2024-04-17T08:01:37.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 1 -- 172.21.15.46:0/747459653 wait complete. 2024-04-17T08:01:37.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.052+0000 7f4ff6372740 20 client.15978 trim_cache size 0 max 16384 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 Processor -- start 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 -- start start 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3fead260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3febcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1b38e0 con 0x55ea3feb0ed0 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1b11a0 con 0x55ea3feab370 2024-04-17T08:01:37.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.071+0000 7f4ff6372740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1aeff0 con 0x55ea3feabd60 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3febcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3febcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44022/0 (socket says 172.21.15.46:44022) 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51786/0 (socket says 172.21.15.46:51786) 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 -- 172.21.15.46:0/3052953155 learned_addr learned my addr 172.21.15.46:0/3052953155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4ff1600640 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3fead260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 -- 172.21.15.46:0/3052953155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 msgr2=0x55ea3fead260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3fead260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 -- 172.21.15.46:0/3052953155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feb0ed0 msgr2=0x55ea3feb5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 -- 172.21.15.46:0/3052953155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3fec39d0 con 0x55ea3feabd60 2024-04-17T08:01:37.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.072+0000 7f4febe00640 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3febcd40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55ea3fec3ec0 tx=0x7f4fd8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1406e7a152c1674 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.073+0000 7f4fdd200640 1 -- 172.21.15.46:0/3052953155 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8004230 con 0x55ea3feabd60 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.073+0000 7f4fdd200640 1 -- 172.21.15.46:0/3052953155 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd8024030 con 0x55ea3feabd60 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.073+0000 7f4fdd200640 1 -- 172.21.15.46:0/3052953155 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8003ab0 con 0x55ea3feabd60 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 -- 172.21.15.46:0/3052953155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 msgr2=0x55ea3febcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3febcd40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55ea3fec3ec0 tx=0x7f4fd8005f90 comp rx=0 tx=0).stop 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 -- 172.21.15.46:0/3052953155 shutdown_connections 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feb0ed0 0x55ea3feb5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feabd60 0x55ea3febcd40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 --2- 172.21.15.46:0/3052953155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3fead260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 -- 172.21.15.46:0/3052953155 >> 172.21.15.46:0/3052953155 conn(0x55ea3fea68e0 msgr2=0x55ea3feaa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 -- 172.21.15.46:0/3052953155 shutdown_connections 2024-04-17T08:01:37.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 -- 172.21.15.46:0/3052953155 wait complete. 2024-04-17T08:01:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.074+0000 7f4ff6372740 1 -- start start 2024-04-17T08:01:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 1 Processor -- start 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3feba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3febae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feb0ed0 0x55ea3febbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1af660 con 0x55ea3feabd60 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1b0ad0 con 0x55ea3feab370 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1afd30 con 0x55ea3feb0ed0 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.075+0000 7f4ff6372740 10 client.0 _ll_register_callbacks cb 0x7f4fd8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feb0ed0 0x55ea3febbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3febae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feb0ed0 0x55ea3febbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44032/0 (socket says 172.21.15.46:44032) 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 -- 172.21.15.46:0/4286459803 learned_addr learned my addr 172.21.15.46:0/4286459803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3feba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 -- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 msgr2=0x55ea3feba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3feba900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 -- 172.21.15.46:0/4286459803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 msgr2=0x55ea3febae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3febae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4ff1600640 1 -- 172.21.15.46:0/4286459803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3febc440 con 0x55ea3feb0ed0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.076+0000 7f4febe00640 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3febae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4ff1600640 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feb0ed0 0x55ea3febbe50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4048860 tx=0x7f4fe4033910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6ccaa481289930 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4fd2000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4fd2000640 1 -- 172.21.15.46:0/4286459803 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4043040 con 0x55ea3feb0ed0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4fd2000640 1 -- 172.21.15.46:0/4286459803 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4fe40348a0 con 0x55ea3feb0ed0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd8024030 con 0x55ea3feb0ed0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4fd2000640 1 -- 172.21.15.46:0/4286459803 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fe402cac0 con 0x55ea3feb0ed0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4ff6372740 10 client.26116 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.077+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea3ff27c60 con 0x55ea3feb0ed0 2024-04-17T08:01:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4fd2000640 1 -- 172.21.15.46:0/4286459803 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd8024030 con 0x55ea3feb0ed0 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4fd2000640 1 -- 172.21.15.46:0/4286459803 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f4fe403f650 con 0x55ea3feb0ed0 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4ff6372740 10 client.26116 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4ff6372740 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4ff6372740 20 client.26116 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4ff6372740 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3ff28570 0x55ea3ff2a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4ff6372740 4 client.26116 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea3fe199b0 con 0x55ea3ff28570 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.078+0000 7f4febe00640 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3ff28570 0x55ea3ff2a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4febe00640 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3ff28570 0x55ea3ff2a990 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2be0ec8ac84481d server_cookie=c384b08c6ab60582 in_seq=0 out_seq=0 2024-04-17T08:01:37.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4fd2000640 10 client.26116 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4fd2000640 1 -- 172.21.15.46:0/4286459803 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea3fe199b0 con 0x55ea3ff28570 2024-04-17T08:01:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4fd2000640 10 client.26116 handle_command_reply: tid=0 2024-04-17T08:01:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4ff6372740 1 client.26116 shutdown 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feb0ed0 msgr2=0x55ea3febbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4ff6372740 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feb0ed0 0x55ea3febbe50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4048860 tx=0x7f4fe4033910 comp rx=0 tx=0).stop 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 shutdown_connections 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.079+0000 7f4ff6372740 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ea3ff28570 0x55ea3ff2a990 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff6372740 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ea3feb0ed0 0x55ea3febbe50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff6372740 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3feabd60 0x55ea3febae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff6372740 1 --2- 172.21.15.46:0/4286459803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ea3feab370 0x55ea3feba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 >> 172.21.15.46:0/4286459803 conn(0x55ea3fea68e0 msgr2=0x55ea3feaa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4fd2000640 0 client.26116 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 shutdown_connections 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff6372740 1 -- 172.21.15.46:0/4286459803 wait complete. 2024-04-17T08:01:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff6372740 20 client.26116 trim_cache size 0 max 16384 2024-04-17T08:01:37.083 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:01:37.083 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-17T08:01:37.083 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-17T08:01:37.083 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4fd40a3bb0 msgr2=0x7f4fd40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff2000640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4fd40a3bb0 0x7f4fd40a6070 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00993a0 tx=0x7f4fe0005c80 comp rx=0 tx=0).stop 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec076920 msgr2=0x7f4fec19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.080+0000 7f4ff2000640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec076920 0x7f4fec19e6b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9c0 tx=0x7f4fe000ee80 comp rx=0 tx=0).stop 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 shutdown_connections 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f4fd40a3bb0 0x7f4fd40a6070 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fec078500 0x7f4fec1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fec077b60 0x7f4fec19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 --2- 172.21.15.46:0/1141555654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fec076920 0x7f4fec19e6b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 >> 172.21.15.46:0/1141555654 conn(0x7f4fec0ff570 msgr2=0x7f4fec101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 shutdown_connections 2024-04-17T08:01:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.081+0000 7f4ff2000640 1 -- 172.21.15.46:0/1141555654 wait complete. 2024-04-17T08:01:37.096 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T08:01:37.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T08:01:37.158 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:36 smithi046 ceph-mon[32038]: pgmap v510: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 835 KiB/s wr, 101 op/s 2024-04-17T08:01:37.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:36 smithi053 ceph-mon[36437]: pgmap v510: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 835 KiB/s wr, 101 op/s 2024-04-17T08:01:37.281 INFO:tasks.workunit.client.0.smithi046.stdout:Building m4 2024-04-17T08:01:37.281 INFO:tasks.workunit.client.0.smithi046.stdout:Building src 2024-04-17T08:01:37.281 INFO:tasks.workunit.client.0.smithi046.stdout:Building ltp 2024-04-17T08:01:37.283 INFO:tasks.workunit.client.0.smithi046.stdout:Building common 2024-04-17T08:01:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 -- 172.21.15.46:0/39913734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b00ffc40 msgr2=0x7f34b0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 --2- 172.21.15.46:0/39913734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b00ffc40 0x7f34b0100040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009900 tx=0x7f34a402bc60 comp rx=0 tx=0).stop 2024-04-17T08:01:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 -- 172.21.15.46:0/39913734 shutdown_connections 2024-04-17T08:01:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 --2- 172.21.15.46:0/39913734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0107d50 0x7f34b010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 --2- 172.21.15.46:0/39913734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b0100610 0x7f34b0107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 --2- 172.21.15.46:0/39913734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b00ffc40 0x7f34b0100040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 -- 172.21.15.46:0/39913734 >> 172.21.15.46:0/39913734 conn(0x7f34b00fb7a0 msgr2=0x7f34b00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 -- 172.21.15.46:0/39913734 shutdown_connections 2024-04-17T08:01:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.291+0000 7f34b7e00640 1 -- 172.21.15.46:0/39913734 wait complete. 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.292+0000 7f34b7e00640 1 Processor -- start 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b7e00640 1 -- start start 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b00ffc40 0x7f34b019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b0100610 0x7f34b019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0107d50 0x7f34b01a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010dc30 con 0x7f34b0107d50 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b7e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010dab0 con 0x7f34b0100610 2024-04-17T08:01:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010ddb0 con 0x7f34b00ffc40 2024-04-17T08:01:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b0100610 0x7f34b019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b4a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b0100610 0x7f34b019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:57058/0 (socket says 172.21.15.46:57058) 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b4a00640 1 -- 172.21.15.46:0/704087326 learned_addr learned my addr 172.21.15.46:0/704087326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b5e00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0107d50 0x7f34b01a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.293+0000 7f34b5400640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b00ffc40 0x7f34b019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b4a00640 1 -- 172.21.15.46:0/704087326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b00ffc40 msgr2=0x7f34b019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b4a00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b00ffc40 0x7f34b019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b4a00640 1 -- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0107d50 msgr2=0x7f34b01a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b4a00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0107d50 0x7f34b01a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b4a00640 1 -- 172.21.15.46:0/704087326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b01a2120 con 0x7f34b0100610 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b4a00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b0100610 0x7f34b019aa40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e9f0 tx=0x7f349c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34ae000640 1 -- 172.21.15.46:0/704087326 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c00cd80 con 0x7f34b0100610 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b01a2410 con 0x7f34b0100610 2024-04-17T08:01:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34ae000640 1 -- 172.21.15.46:0/704087326 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349c010910 con 0x7f34b0100610 2024-04-17T08:01:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34ae000640 1 -- 172.21.15.46:0/704087326 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0188f0 con 0x7f34b0100610 2024-04-17T08:01:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.294+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01a29b0 con 0x7f34b0100610 2024-04-17T08:01:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.296+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b01a2cd0 con 0x7f34b0100610 2024-04-17T08:01:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.298+0000 7f34ae000640 1 -- 172.21.15.46:0/704087326 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f349c010ac0 con 0x7f34b0100610 2024-04-17T08:01:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.299+0000 7f34ae000640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34980a3bb0 0x7f34980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.300+0000 7f34ae000640 1 -- 172.21.15.46:0/704087326 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f349c0d0f70 con 0x7f34b0100610 2024-04-17T08:01:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.300+0000 7f34b5400640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34980a3bb0 0x7f34980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.301+0000 7f34b5400640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34980a3bb0 0x7f34980a6070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006010 tx=0x7f34a402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:36 smithi062 ceph-mon[36507]: pgmap v510: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 835 KiB/s wr, 101 op/s 2024-04-17T08:01:37.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.305+0000 7f34ae000640 1 -- 172.21.15.46:0/704087326 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f349c09cf10 con 0x7f34b0100610 2024-04-17T08:01:37.305 INFO:tasks.workunit.client.0.smithi046.stdout: [DEP] 2024-04-17T08:01:37.320 INFO:tasks.workunit.client.0.smithi046.stdout: [DEP] 2024-04-17T08:01:37.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.562+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34b01a2ea0 con 0x7f34b0100610 2024-04-17T08:01:37.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.563+0000 7f34ae000640 1 -- 172.21.15.46:0/704087326 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11872 (secure 0 0 0) 0x7f349c0a1210 con 0x7f34b0100610 2024-04-17T08:01:37.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:01:37.562 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T08:00:35.051221+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T08:01:37.563 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-04-17T08:01:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34980a3bb0 msgr2=0x7f34980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34980a3bb0 0x7f34980a6070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006010 tx=0x7f34a402df90 comp rx=0 tx=0).stop 2024-04-17T08:01:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b0100610 msgr2=0x7f34b019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b0100610 0x7f34b019aa40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e9f0 tx=0x7f349c00eeb0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 shutdown_connections 2024-04-17T08:01:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34980a3bb0 0x7f34980a6070 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34b0107d50 0x7f34b01a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34b0100610 0x7f34b019aa40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 --2- 172.21.15.46:0/704087326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b00ffc40 0x7f34b019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 >> 172.21.15.46:0/704087326 conn(0x7f34b00fb7a0 msgr2=0x7f34b00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 shutdown_connections 2024-04-17T08:01:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.566+0000 7f34b7e00640 1 -- 172.21.15.46:0/704087326 wait complete. 2024-04-17T08:01:37.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T08:01:37.785 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] doio 2024-04-17T08:01:37.785 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] fsstress 2024-04-17T08:01:37.786 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] fsx 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.810+0000 7f8bd6200640 1 -- 172.21.15.46:0/3013276833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd01031a0 msgr2=0x7f8bd0105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.810+0000 7f8bd6200640 1 --2- 172.21.15.46:0/3013276833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd01031a0 0x7f8bd0105610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009860 tx=0x7f8bbc02bb70 comp rx=0 tx=0).stop 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.811+0000 7f8bd6200640 1 -- 172.21.15.46:0/3013276833 shutdown_connections 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.811+0000 7f8bd6200640 1 --2- 172.21.15.46:0/3013276833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd0105b50 0x7f8bd0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.811+0000 7f8bd6200640 1 --2- 172.21.15.46:0/3013276833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd01031a0 0x7f8bd0105610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.811+0000 7f8bd6200640 1 --2- 172.21.15.46:0/3013276833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd0100830 0x7f8bd0102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.811+0000 7f8bd6200640 1 -- 172.21.15.46:0/3013276833 >> 172.21.15.46:0/3013276833 conn(0x7f8bd00fc430 msgr2=0x7f8bd00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.811+0000 7f8bd6200640 1 -- 172.21.15.46:0/3013276833 shutdown_connections 2024-04-17T08:01:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.811+0000 7f8bd6200640 1 -- 172.21.15.46:0/3013276833 wait complete. 2024-04-17T08:01:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.812+0000 7f8bd6200640 1 Processor -- start 2024-04-17T08:01:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.812+0000 7f8bd6200640 1 -- start start 2024-04-17T08:01:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.812+0000 7f8bd6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd0100830 0x7f8bd019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.812+0000 7f8bd6200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd01031a0 0x7f8bd019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.812+0000 7f8bd6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd0105b50 0x7f8bd01a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.812+0000 7f8bd6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0114220 con 0x7f8bd0105b50 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bd6200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01140a0 con 0x7f8bd01031a0 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bd6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01143a0 con 0x7f8bd0100830 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd0100830 0x7f8bd019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd0100830 0x7f8bd019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44070/0 (socket says 172.21.15.46:44070) 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 -- 172.21.15.46:0/1515567501 learned_addr learned my addr 172.21.15.46:0/1515567501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcea00640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd01031a0 0x7f8bd019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd0105b50 0x7f8bd01a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 -- 172.21.15.46:0/1515567501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd01031a0 msgr2=0x7f8bd019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd01031a0 0x7f8bd019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 -- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd0105b50 msgr2=0x7f8bd01a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd0105b50 0x7f8bd01a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcf400640 1 -- 172.21.15.46:0/1515567501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd01a64e0 con 0x7f8bd0100830 2024-04-17T08:01:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcea00640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd01031a0 0x7f8bd019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.813+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd0105b50 0x7f8bd01a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.814+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd0100830 0x7f8bd019e8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e9c0 tx=0x7f8bc000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.814+0000 7f8bc7e00640 1 -- 172.21.15.46:0/1515567501 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000ccc0 con 0x7f8bd0100830 2024-04-17T08:01:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.814+0000 7f8bc7e00640 1 -- 172.21.15.46:0/1515567501 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0005500 con 0x7f8bd0100830 2024-04-17T08:01:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.814+0000 7f8bc7e00640 1 -- 172.21.15.46:0/1515567501 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0010430 con 0x7f8bd0100830 2024-04-17T08:01:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.814+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd01a67d0 con 0x7f8bd0100830 2024-04-17T08:01:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.814+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd01a6c40 con 0x7f8bd0100830 2024-04-17T08:01:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.819+0000 7f8bc7e00640 1 -- 172.21.15.46:0/1515567501 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bc00046c0 con 0x7f8bd0100830 2024-04-17T08:01:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.819+0000 7f8bd8e64740 1 Processor -- start 2024-04-17T08:01:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.820+0000 7f8bd8e64740 1 -- start start 2024-04-17T08:01:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.820+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.820+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 0x55d18a9cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.820+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.820+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d18a92d020 con 0x55d18a9c7990 2024-04-17T08:01:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.820+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d18a8d3460 con 0x55d18a9c6fa0 2024-04-17T08:01:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.820+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d18a928fe0 con 0x55d18a9ccb00 2024-04-17T08:01:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcfe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:44072/0 (socket says 172.21.15.46:44072) 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcea00640 1 -- 172.21.15.46:0/3849585125 learned_addr learned my addr 172.21.15.46:0/3849585125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcf400640 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 0x55d18a9cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bc7e00640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8bb40a3bb0 0x7f8bb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcfe00640 1 -- 172.21.15.46:0/3849585125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 msgr2=0x55d18a9cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcfe00640 1 -- 172.21.15.46:0/3849585125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 msgr2=0x55d18a9cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 0x55d18a9cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bcfe00640 1 -- 172.21.15.46:0/3849585125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18a8db040 con 0x55d18a9c6fa0 2024-04-17T08:01:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.821+0000 7f8bc7e00640 1 -- 172.21.15.46:0/1515567501 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8bc00d0ae0 con 0x7f8bd0100830 2024-04-17T08:01:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.822+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c73c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ec0 tx=0x7f8bc800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae2279f3fa9e616 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.822+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8bb40a3bb0 0x7f8bb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.823+0000 7f8bbbe00640 1 -- 172.21.15.46:0/3849585125 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8011640 con 0x55d18a9c6fa0 2024-04-17T08:01:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.823+0000 7f8bbbe00640 1 -- 172.21.15.46:0/3849585125 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc801ddc0 con 0x55d18a9c6fa0 2024-04-17T08:01:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.823+0000 7f8bbbe00640 1 -- 172.21.15.46:0/3849585125 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8010d90 con 0x55d18a9c6fa0 2024-04-17T08:01:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.823+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8bb40a3bb0 0x7f8bb40a6070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8bc801e000 tx=0x7f8bc8021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.824+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3849585125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 msgr2=0x55d18a9c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.824+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c73c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ec0 tx=0x7f8bc800f730 comp rx=0 tx=0).stop 2024-04-17T08:01:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.824+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3849585125 shutdown_connections 2024-04-17T08:01:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.824+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.824+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 0x55d18a9cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.824+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3849585125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c73c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.824+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3849585125 >> 172.21.15.46:0/3849585125 conn(0x55d18a9c2510 msgr2=0x55d18a9c2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.825+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3849585125 shutdown_connections 2024-04-17T08:01:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.825+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3849585125 wait complete. 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.825+0000 7f8bd8e64740 1 -- start start 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.825+0000 7f8bd8e64740 1 Processor -- start 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c7990 0x55d18a9d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d18a9cfab0 con 0x55d18a9ccb00 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d18a8e6830 con 0x55d18a9c7990 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d18a9cfc30 con 0x55d18a9c6fa0 2024-04-17T08:01:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bd8e64740 10 client.0 _ll_register_callbacks cb 0x55d18a8b4fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c7990 0x55d18a9d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bcf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c7990 0x55d18a9d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46684/0 (socket says 172.21.15.46:46684) 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.826+0000 7f8bcea00640 1 -- 172.21.15.46:0/1795346317 learned_addr learned my addr 172.21.15.46:0/1795346317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bcea00640 1 -- 172.21.15.46:0/1795346317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c6fa0 msgr2=0x55d18a9d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bcea00640 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bcea00640 1 -- 172.21.15.46:0/1795346317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 msgr2=0x55d18a9d8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bcea00640 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bcea00640 1 -- 172.21.15.46:0/1795346317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18a8ea920 con 0x55d18a9c7990 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bcea00640 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c7990 0x55d18a9d1a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc037040 tx=0x7f8bbc005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eadac381d54bcc9e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.827+0000 7f8bb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.828+0000 7f8bb8c00640 1 -- 172.21.15.46:0/1795346317 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc045070 con 0x55d18a9c7990 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.828+0000 7f8bb8c00640 1 -- 172.21.15.46:0/1795346317 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8bbc004240 con 0x55d18a9c7990 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.828+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc801ddc0 con 0x55d18a9c7990 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.829+0000 7f8bb8c00640 1 -- 172.21.15.46:0/1795346317 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc04f050 con 0x55d18a9c7990 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.829+0000 7f8bd8e64740 10 client.26162 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.829+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d18a8b87f0 con 0x55d18a9c7990 2024-04-17T08:01:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.829+0000 7f8bb8c00640 1 -- 172.21.15.46:0/1795346317 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc801ddc0 con 0x55d18a9c7990 2024-04-17T08:01:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.830+0000 7f8bb8c00640 1 -- 172.21.15.46:0/1795346317 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f8bbc02c510 con 0x55d18a9c7990 2024-04-17T08:01:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.830+0000 7f8bd8e64740 10 client.26162 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.831+0000 7f8bd8e64740 10 client.26162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:01:37.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.831+0000 7f8bd8e64740 20 client.26162 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.831+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18a9dcf50 0x55d18a9df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.831+0000 7f8bd8e64740 4 client.26162 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.831+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d18a93aa40 con 0x55d18a9dcf50 2024-04-17T08:01:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.831+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18a9dcf50 0x55d18a9df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.832+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18a9dcf50 0x55d18a9df3f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2aab5a2b04acc07f server_cookie=a45da5696eb982c9 in_seq=0 out_seq=0 2024-04-17T08:01:37.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.832+0000 7f8bb8c00640 10 client.26162 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:37.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.832+0000 7f8bb8c00640 1 -- 172.21.15.46:0/1795346317 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d18a93aa40 con 0x55d18a9dcf50 2024-04-17T08:01:37.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.832+0000 7f8bb8c00640 10 client.26162 handle_command_reply: tid=0 2024-04-17T08:01:37.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.833+0000 7f8bd8e64740 1 client.26162 shutdown 2024-04-17T08:01:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.833+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c7990 msgr2=0x55d18a9d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.833+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c7990 0x55d18a9d1a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc037040 tx=0x7f8bbc005ad0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 shutdown_connections 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18a9dcf50 0x55d18a9df3f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c7990 0x55d18a9d1a20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1795346317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 >> 172.21.15.46:0/1795346317 conn(0x55d18a9c2510 msgr2=0x55d18a9c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bb8c00640 0 client.26162 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 shutdown_connections 2024-04-17T08:01:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1795346317 wait complete. 2024-04-17T08:01:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.834+0000 7f8bd8e64740 20 client.26162 trim_cache size 0 max 16384 2024-04-17T08:01:37.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.856+0000 7f8bd8e64740 1 Processor -- start 2024-04-17T08:01:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bd8e64740 1 -- start start 2024-04-17T08:01:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 0x55d18a9d8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01b3ae0 con 0x55d18a9c7990 2024-04-17T08:01:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01b13a0 con 0x55d18a9c6fa0 2024-04-17T08:01:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01af1f0 con 0x55d18a9ccb00 2024-04-17T08:01:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bcf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bcf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46694/0 (socket says 172.21.15.46:46694) 2024-04-17T08:01:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.857+0000 7f8bcf400640 1 -- 172.21.15.46:0/1712061187 learned_addr learned my addr 172.21.15.46:0/1712061187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.858+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.858+0000 7f8bcf400640 1 -- 172.21.15.46:0/1712061187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 msgr2=0x55d18a9c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.858+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.858+0000 7f8bcf400640 1 -- 172.21.15.46:0/1712061187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 msgr2=0x55d18a9d8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.858+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 0x55d18a9d8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.858+0000 7f8bcf400640 1 -- 172.21.15.46:0/1712061187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18a63f3f0 con 0x55d18a9c6fa0 2024-04-17T08:01:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.858+0000 7f8bcf400640 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c8e90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8bc009c9c0 tx=0x7f8bc00ce120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f707af08d6ce0321 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.859+0000 7f8bbb400640 1 -- 172.21.15.46:0/1712061187 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc009ca30 con 0x55d18a9c6fa0 2024-04-17T08:01:37.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.859+0000 7f8bbb400640 1 -- 172.21.15.46:0/1712061187 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc00a0de0 con 0x55d18a9c6fa0 2024-04-17T08:01:37.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.859+0000 7f8bbb400640 1 -- 172.21.15.46:0/1712061187 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00a5210 con 0x55d18a9c6fa0 2024-04-17T08:01:37.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.860+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1712061187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 msgr2=0x55d18a9c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.860+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c8e90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8bc009c9c0 tx=0x7f8bc00ce120 comp rx=0 tx=0).stop 2024-04-17T08:01:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.860+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1712061187 shutdown_connections 2024-04-17T08:01:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.860+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9ccb00 0x55d18a9c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.860+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9c7990 0x55d18a9d8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.860+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/1712061187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9c8e90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.860+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1712061187 >> 172.21.15.46:0/1712061187 conn(0x55d18a9c2510 msgr2=0x55d18a9cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.861+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1712061187 shutdown_connections 2024-04-17T08:01:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.862+0000 7f8bd8e64740 1 -- 172.21.15.46:0/1712061187 wait complete. 2024-04-17T08:01:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.862+0000 7f8bd8e64740 1 -- start start 2024-04-17T08:01:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.862+0000 7f8bd8e64740 1 Processor -- start 2024-04-17T08:01:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c7990 0x55d18a9d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bd8e64740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01af860 con 0x55d18a9ccb00 2024-04-17T08:01:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01b0cd0 con 0x55d18a9c6fa0 2024-04-17T08:01:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bd8e64740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd01aff30 con 0x55d18a9c7990 2024-04-17T08:01:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bd8e64740 10 client.0 _ll_register_callbacks cb 0x7f8bbc034800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bcf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bcf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46696/0 (socket says 172.21.15.46:46696) 2024-04-17T08:01:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.863+0000 7f8bcf400640 1 -- 172.21.15.46:0/3893505472 learned_addr learned my addr 172.21.15.46:0/3893505472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcea00640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c7990 0x55d18a9d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcfe00640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcea00640 1 -- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 msgr2=0x55d18a9d5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcea00640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcea00640 1 -- 172.21.15.46:0/3893505472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 msgr2=0x55d18a9d7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcea00640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcea00640 1 -- 172.21.15.46:0/3893505472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18a7097d0 con 0x55d18a9c7990 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcf400640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bcea00640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c7990 0x55d18a9d6430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55d18a9dbb90 tx=0x7f8bbc02c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6003934b9569e10e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.864+0000 7f8bb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.865+0000 7f8bb8c00640 1 -- 172.21.15.46:0/3893505472 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc045070 con 0x55d18a9c7990 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.865+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc00a0de0 con 0x55d18a9c7990 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.865+0000 7f8bb8c00640 1 -- 172.21.15.46:0/3893505472 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f8bbc004220 con 0x55d18a9c7990 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.865+0000 7f8bb8c00640 1 -- 172.21.15.46:0/3893505472 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc039040 con 0x55d18a9c7990 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.865+0000 7f8bd8e64740 10 client.26140 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.865+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d18a9d8b60 con 0x55d18a9c7990 2024-04-17T08:01:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bb8c00640 1 -- 172.21.15.46:0/3893505472 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc00a0de0 con 0x55d18a9c7990 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bb8c00640 1 -- 172.21.15.46:0/3893505472 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f8bbc0035f0 con 0x55d18a9c7990 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bd8e64740 10 client.26140 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bd8e64740 10 client.26140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bd8e64740 20 client.26140 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18aa43ad0 0x55d18aa45ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bd8e64740 4 client.26140 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.866+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d18a9348b0 con 0x55d18aa43ad0 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.867+0000 7f8bcf400640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18aa43ad0 0x55d18aa45ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.867+0000 7f8bcf400640 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18aa43ad0 0x55d18aa45ef0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f810533c0f69601 server_cookie=cbff63b527dbd30f in_seq=0 out_seq=0 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.867+0000 7f8bb8c00640 10 client.26140 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bb8c00640 1 -- 172.21.15.46:0/3893505472 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d18a9348b0 con 0x55d18aa43ad0 2024-04-17T08:01:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bb8c00640 10 client.26140 handle_command_reply: tid=0 2024-04-17T08:01:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 client.26140 shutdown 2024-04-17T08:01:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c7990 msgr2=0x55d18a9d6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c7990 0x55d18a9d6430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55d18a9dbb90 tx=0x7f8bbc02c7c0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 shutdown_connections 2024-04-17T08:01:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x55d18aa43ad0 0x55d18aa45ef0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d18a9ccb00 0x55d18a9d7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d18a9c7990 0x55d18a9d6430 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bb8c00640 0 client.26140 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 --2- 172.21.15.46:0/3893505472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d18a9c6fa0 0x55d18a9d5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 >> 172.21.15.46:0/3893505472 conn(0x55d18a9c2510 msgr2=0x55d18a9d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 shutdown_connections 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 1 -- 172.21.15.46:0/3893505472 wait complete. 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.868+0000 7f8bd8e64740 20 client.26140 trim_cache size 0 max 16384 2024-04-17T08:01:37.869 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:01:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.869+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8bb40a3bb0 msgr2=0x7f8bb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.869+0000 7f8bd6200640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8bb40a3bb0 0x7f8bb40a6070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8bc801e000 tx=0x7f8bc8021000 comp rx=0 tx=0).stop 2024-04-17T08:01:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.869+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd0100830 msgr2=0x7f8bd019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.869+0000 7f8bd6200640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd0100830 0x7f8bd019e8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e9c0 tx=0x7f8bc000ee80 comp rx=0 tx=0).stop 2024-04-17T08:01:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.870+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 shutdown_connections 2024-04-17T08:01:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.870+0000 7f8bd6200640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f8bb40a3bb0 0x7f8bb40a6070 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.870+0000 7f8bd6200640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bd0105b50 0x7f8bd01a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.870+0000 7f8bd6200640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd01031a0 0x7f8bd019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.870+0000 7f8bd6200640 1 --2- 172.21.15.46:0/1515567501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bd0100830 0x7f8bd019e8c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.870+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 >> 172.21.15.46:0/1515567501 conn(0x7f8bd00fc430 msgr2=0x7f8bd01011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.871+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 shutdown_connections 2024-04-17T08:01:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:37.871+0000 7f8bd6200640 1 -- 172.21.15.46:0/1515567501 wait complete. 2024-04-17T08:01:37.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T08:01:38.130 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c: In function 'attr_list_path': 2024-04-17T08:01:38.130 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~ 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr:In file included from ../src/global.h:25, 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: from fsstress.c:12: 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~ 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c: In function 'attr_remove_path': 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~ 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr:In file included from ../src/global.h:25, 2024-04-17T08:01:38.131 INFO:tasks.workunit.client.0.smithi046.stderr: from fsstress.c:12: 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~ 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c: In function 'attr_set_path': 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~ 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr:In file included from ../src/global.h:25, 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr: from fsstress.c:12: 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-04-17T08:01:38.132 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~ 2024-04-17T08:01:38.143 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c: In function 'allocsp_f': 2024-04-17T08:01:38.144 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-17T08:01:38.144 INFO:tasks.workunit.client.0.smithi046.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-04-17T08:01:38.144 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-17T08:01:38.144 INFO:tasks.workunit.client.0.smithi046.stderr: | XFS_IOC_RESVSP64 2024-04-17T08:01:38.144 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-04-17T08:01:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 -- 172.21.15.46:0/2470552686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0103b60 msgr2=0x7f13b010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 --2- 172.21.15.46:0/2470552686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0103b60 0x7f13b010a3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009be0 tx=0x7f13a802bef0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 -- 172.21.15.46:0/2470552686 shutdown_connections 2024-04-17T08:01:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 --2- 172.21.15.46:0/2470552686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0103b60 0x7f13b010a3f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 --2- 172.21.15.46:0/2470552686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b01031a0 0x7f13b0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 --2- 172.21.15.46:0/2470552686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b0101fb0 0x7f13b01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 -- 172.21.15.46:0/2470552686 >> 172.21.15.46:0/2470552686 conn(0x7f13b00fd520 msgr2=0x7f13b00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.153+0000 7f13b8000640 1 -- 172.21.15.46:0/2470552686 shutdown_connections 2024-04-17T08:01:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.154+0000 7f13b8000640 1 -- 172.21.15.46:0/2470552686 wait complete. 2024-04-17T08:01:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.155+0000 7f13b8000640 1 Processor -- start 2024-04-17T08:01:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.155+0000 7f13b8000640 1 -- start start 2024-04-17T08:01:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b8000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b0101fb0 0x7f13b019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b8000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b01031a0 0x7f13b019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b8000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b0103b60 0x7f13b01a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b8000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010dc50 con 0x7f13b0103b60 2024-04-17T08:01:38.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b8000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010dad0 con 0x7f13b0101fb0 2024-04-17T08:01:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b8000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010ddd0 con 0x7f13b01031a0 2024-04-17T08:01:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b5600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b0101fb0 0x7f13b019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b5600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b0101fb0 0x7f13b019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46722/0 (socket says 172.21.15.46:46722) 2024-04-17T08:01:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b5600640 1 -- 172.21.15.46:0/2132829675 learned_addr learned my addr 172.21.15.46:0/2132829675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.156+0000 7f13b6000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b0103b60 0x7f13b01a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.157+0000 7f13b4c00640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b01031a0 0x7f13b019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.157+0000 7f13b5600640 1 -- 172.21.15.46:0/2132829675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b01031a0 msgr2=0x7f13b019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.157+0000 7f13b5600640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b01031a0 0x7f13b019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.157+0000 7f13b5600640 1 -- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b0103b60 msgr2=0x7f13b01a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.157+0000 7f13b5600640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b0103b60 0x7f13b01a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.157+0000 7f13b5600640 1 -- 172.21.15.46:0/2132829675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b01a2120 con 0x7f13b0101fb0 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.157+0000 7f13b5600640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b0101fb0 0x7f13b019a500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e9c0 tx=0x7f13a400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.158+0000 7f13ae000640 1 -- 172.21.15.46:0/2132829675 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4024070 con 0x7f13b0101fb0 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.158+0000 7f13ae000640 1 -- 172.21.15.46:0/2132829675 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a401f440 con 0x7f13b0101fb0 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.158+0000 7f13ae000640 1 -- 172.21.15.46:0/2132829675 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a401d5a0 con 0x7f13b0101fb0 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.158+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b0104420 con 0x7f13b0101fb0 2024-04-17T08:01:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.158+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b0104920 con 0x7f13b0101fb0 2024-04-17T08:01:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.162+0000 7f13babf4740 1 Processor -- start 2024-04-17T08:01:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.162+0000 7f13babf4740 1 -- start start 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13babf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13babf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509de490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13babf4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13babf4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55775093f020 con 0x5577509deb00 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13babf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5577508e5460 con 0x5577509d9990 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13babf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55775093afe0 con 0x5577509d8fa0 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13b6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13b6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509d93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58538/0 (socket says 172.21.15.46:58538) 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13b6000640 1 -- 172.21.15.46:0/1807024280 learned_addr learned my addr 172.21.15.46:0/1807024280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.163+0000 7f13b4c00640 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13b6000640 1 -- 172.21.15.46:0/1807024280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 msgr2=0x5577509de490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T08:01:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13b6000640 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509de490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13b6000640 1 -- 172.21.15.46:0/1807024280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 msgr2=0x5577509e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13b6000640 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13b6000640 1 -- 172.21.15.46:0/1807024280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577508ed040 con 0x5577509d8fa0 2024-04-17T08:01:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13ae000640 1 -- 172.21.15.46:0/2132829675 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f13a401f5f0 con 0x7f13b0101fb0 2024-04-17T08:01:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13b6000640 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509d93c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009be0 tx=0x7f13a8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e3c490436e815d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13a2600640 1 -- 172.21.15.46:0/1807024280 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8043070 con 0x5577509d8fa0 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13a2600640 1 -- 172.21.15.46:0/1807024280 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a8033bc0 con 0x5577509d8fa0 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.164+0000 7f13a2600640 1 -- 172.21.15.46:0/1807024280 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a802caf0 con 0x5577509d8fa0 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.165+0000 7f13ae000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f13980a3c30 0x7f13980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13babf4740 1 -- 172.21.15.46:0/1807024280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 msgr2=0x5577509d93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13babf4740 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509d93c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009be0 tx=0x7f13a8009310 comp rx=0 tx=0).stop 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13ae000640 1 -- 172.21.15.46:0/2132829675 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f13a40d8d30 con 0x7f13b0101fb0 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13b5600640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f13980a3c30 0x7f13980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13babf4740 1 -- 172.21.15.46:0/1807024280 shutdown_connections 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13babf4740 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13babf4740 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509de490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13babf4740 1 --2- 172.21.15.46:0/1807024280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509d93c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.166+0000 7f13babf4740 1 -- 172.21.15.46:0/1807024280 >> 172.21.15.46:0/1807024280 conn(0x5577509d4510 msgr2=0x5577509d4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.167+0000 7f13babf4740 1 -- 172.21.15.46:0/1807024280 shutdown_connections 2024-04-17T08:01:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.167+0000 7f13babf4740 1 -- 172.21.15.46:0/1807024280 wait complete. 2024-04-17T08:01:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.167+0000 7f13b5600640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f13980a3c30 0x7f13980a60f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f13a40d6440 tx=0x7f13a400efb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.167+0000 7f13babf4740 1 -- start start 2024-04-17T08:01:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 1 Processor -- start 2024-04-17T08:01:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.169 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c: In function 'freesp_f': 2024-04-17T08:01:38.170 INFO:tasks.workunit.client.0.smithi046.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-17T08:01:38.170 INFO:tasks.workunit.client.0.smithi046.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-04-17T08:01:38.170 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-17T08:01:38.170 INFO:tasks.workunit.client.0.smithi046.stderr: | XFS_IOC_RESVSP64 2024-04-17T08:01:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d9990 0x5577509e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x5577509ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5577509e1a50 con 0x5577509d9990 2024-04-17T08:01:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5577508f8830 con 0x5577509deb00 2024-04-17T08:01:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5577509e1bd0 con 0x5577509d8fa0 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.168+0000 7f13babf4740 10 client.0 _ll_register_callbacks cb 0x5577508c6fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d9990 0x5577509e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d9990 0x5577509e3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54818/0 (socket says 172.21.15.46:54818) 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 -- 172.21.15.46:0/2130281014 learned_addr learned my addr 172.21.15.46:0/2130281014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b6000640 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x5577509ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b6000640 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 -- 172.21.15.46:0/2130281014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 msgr2=0x5577509e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509e3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 -- 172.21.15.46:0/2130281014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 msgr2=0x5577509ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x5577509ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.169+0000 7f13b4c00640 1 -- 172.21.15.46:0/2130281014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577508fc920 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.170+0000 7f13b4c00640 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d9990 0x5577509e3960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f139c012700 tx=0x7f139c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63b38e87b4034d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.170+0000 7f1396a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.170+0000 7f1396a00640 1 -- 172.21.15.46:0/2130281014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c010d60 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.170+0000 7f1396a00640 1 -- 172.21.15.46:0/2130281014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f139c01c3f0 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.170+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a8033bc0 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f1396a00640 1 -- 172.21.15.46:0/2130281014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f139c027050 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 10 client.16011 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577508ca7f0 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f1396a00640 1 -- 172.21.15.46:0/2130281014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a8033bc0 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f1396a00640 1 -- 172.21.15.46:0/2130281014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f139c00e680 con 0x5577509d9990 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 10 client.16011 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 10 client.16011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 20 client.16011 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5577509ef010 0x5577509f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 4 client.16011 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.171+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55775094ca40 con 0x5577509ef010 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.172+0000 7f13b6000640 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5577509ef010 0x5577509f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.173+0000 7f13b6000640 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5577509ef010 0x5577509f1450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ff0bae12b39352a server_cookie=9daa957addcf93d3 in_seq=0 out_seq=0 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.173+0000 7f1396a00640 10 client.16011 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.173+0000 7f1396a00640 1 -- 172.21.15.46:0/2130281014 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55775094ca40 con 0x5577509ef010 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.173+0000 7f1396a00640 10 client.16011 handle_command_reply: tid=0 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 client.16011 shutdown 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d9990 msgr2=0x5577509e3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d9990 0x5577509e3960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f139c012700 tx=0x7f139c012bc0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 shutdown_connections 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x5577509ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x5577509ef010 0x5577509f1450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d9990 0x5577509e3960 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 --2- 172.21.15.46:0/2130281014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 >> 172.21.15.46:0/2130281014 conn(0x5577509d4510 msgr2=0x5577509d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.174+0000 7f1396a00640 0 client.16011 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.175+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 shutdown_connections 2024-04-17T08:01:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.175+0000 7f13babf4740 1 -- 172.21.15.46:0/2130281014 wait complete. 2024-04-17T08:01:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.175+0000 7f13babf4740 20 client.16011 trim_cache size 0 max 16384 2024-04-17T08:01:38.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 Processor -- start 2024-04-17T08:01:38.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 -- start start 2024-04-17T08:01:38.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01ab400 con 0x5577509deb00 2024-04-17T08:01:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01a8cc0 con 0x5577509d9990 2024-04-17T08:01:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.191+0000 7f13babf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01a6b10 con 0x5577509d8fa0 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.192+0000 7f13b6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.192+0000 7f13b6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58542/0 (socket says 172.21.15.46:58542) 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.192+0000 7f13b6000640 1 -- 172.21.15.46:0/3468942566 learned_addr learned my addr 172.21.15.46:0/3468942566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.192+0000 7f13b4c00640 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.192+0000 7f13b6000640 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.192+0000 7f13b4c00640 1 -- 172.21.15.46:0/3468942566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 msgr2=0x5577509dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.193+0000 7f13b4c00640 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.193+0000 7f13b4c00640 1 -- 172.21.15.46:0/3468942566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 msgr2=0x5577509ea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.193+0000 7f13b4c00640 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509ea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.193+0000 7f13b4c00640 1 -- 172.21.15.46:0/3468942566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577506513f0 con 0x5577509d9990 2024-04-17T08:01:38.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.193+0000 7f13b6000640 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:38.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.193+0000 7f13b4c00640 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509db390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f139c00ece0 tx=0x7f139c01b3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e0729929859006b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.194+0000 7f13a1c00640 1 -- 172.21.15.46:0/3468942566 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c031020 con 0x5577509d9990 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.194+0000 7f13a1c00640 1 -- 172.21.15.46:0/3468942566 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f139c004040 con 0x5577509d9990 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.194+0000 7f13a1c00640 1 -- 172.21.15.46:0/3468942566 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c005040 con 0x5577509d9990 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.194+0000 7f13babf4740 1 -- 172.21.15.46:0/3468942566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 msgr2=0x5577509db390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.194+0000 7f13babf4740 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509db390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f139c00ece0 tx=0x7f139c01b3a0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 -- 172.21.15.46:0/3468942566 shutdown_connections 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509deb00 0x5577509ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509d9990 0x5577509db390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 --2- 172.21.15.46:0/3468942566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d8fa0 0x5577509dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 -- 172.21.15.46:0/3468942566 >> 172.21.15.46:0/3468942566 conn(0x5577509d4510 msgr2=0x5577509e3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 -- 172.21.15.46:0/3468942566 shutdown_connections 2024-04-17T08:01:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 -- 172.21.15.46:0/3468942566 wait complete. 2024-04-17T08:01:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.195+0000 7f13babf4740 1 -- start start 2024-04-17T08:01:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 1 Processor -- start 2024-04-17T08:01:38.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d8fa0 0x5577509e7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d9990 0x5577509e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x557750a535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01a7180 con 0x5577509d8fa0 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01a85f0 con 0x5577509deb00 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01a7850 con 0x5577509d9990 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.196+0000 7f13babf4740 10 client.0 _ll_register_callbacks cb 0x7f139c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d8fa0 0x5577509e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d9990 0x5577509e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d8fa0 0x5577509e7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54848/0 (socket says 172.21.15.46:54848) 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b6000640 1 -- 172.21.15.46:0/2558150064 learned_addr learned my addr 172.21.15.46:0/2558150064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b4c00640 1 -- 172.21.15.46:0/2558150064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 msgr2=0x557750a535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b6000640 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x557750a535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b4c00640 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x557750a535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b4c00640 1 -- 172.21.15.46:0/2558150064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d8fa0 msgr2=0x5577509e7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b4c00640 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d8fa0 0x5577509e7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b4c00640 1 -- 172.21.15.46:0/2558150064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775071b7d0 con 0x5577509d9990 2024-04-17T08:01:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.197+0000 7f13b6000640 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x557750a535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.198+0000 7f13b4c00640 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d9990 0x5577509e7d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f139c0126d0 tx=0x7f139c009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d67131b9ecc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.198+0000 7f1396a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.198+0000 7f1396a00640 1 -- 172.21.15.46:0/2558150064 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c031070 con 0x5577509d9990 2024-04-17T08:01:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.198+0000 7f1396a00640 1 -- 172.21.15.46:0/2558150064 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f139c0043f0 con 0x5577509d9990 2024-04-17T08:01:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.198+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f139c004040 con 0x5577509d9990 2024-04-17T08:01:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f1396a00640 1 -- 172.21.15.46:0/2558150064 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f139c010b60 con 0x5577509d9990 2024-04-17T08:01:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 10 client.26158 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557750a54b90 con 0x5577509d9990 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f1396a00640 1 -- 172.21.15.46:0/2558150064 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f139c004040 con 0x5577509d9990 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f1396a00640 1 -- 172.21.15.46:0/2558150064 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f139c004840 con 0x5577509d9990 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 10 client.26158 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 10 client.26158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 20 client.26158 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x557750a55470 0x557750a57890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 4 client.26158 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.199+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577509e3210 con 0x557750a55470 2024-04-17T08:01:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.200+0000 7f13b6000640 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x557750a55470 0x557750a57890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13b6000640 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x557750a55470 0x557750a57890 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=379fcb93595e0f7e server_cookie=6996523e718f0d4 in_seq=0 out_seq=0 2024-04-17T08:01:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f1396a00640 10 client.26158 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f1396a00640 1 -- 172.21.15.46:0/2558150064 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577509e3210 con 0x557750a55470 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f1396a00640 10 client.26158 handle_command_reply: tid=0 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 client.26158 shutdown 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d9990 msgr2=0x5577509e7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d9990 0x5577509e7d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f139c0126d0 tx=0x7f139c009c50 comp rx=0 tx=0).stop 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 shutdown_connections 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5577509deb00 0x557750a535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5577509d9990 0x5577509e7d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x557750a55470 0x557750a57890 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 --2- 172.21.15.46:0/2558150064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577509d8fa0 0x5577509e7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f1396a00640 0 client.26158 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.201+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 >> 172.21.15.46:0/2558150064 conn(0x5577509d4510 msgr2=0x5577509e1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.202+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 shutdown_connections 2024-04-17T08:01:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.202+0000 7f13babf4740 1 -- 172.21.15.46:0/2558150064 wait complete. 2024-04-17T08:01:38.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.202+0000 7f13babf4740 20 client.26158 trim_cache size 0 max 16384 2024-04-17T08:01:38.203 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f13980a3c30 msgr2=0x7f13980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f13980a3c30 0x7f13980a60f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f13a40d6440 tx=0x7f13a400efb0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b0101fb0 msgr2=0x7f13b019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b0101fb0 0x7f13b019a500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e9c0 tx=0x7f13a400ee80 comp rx=0 tx=0).stop 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 shutdown_connections 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f13980a3c30 0x7f13980a60f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b0103b60 0x7f13b01a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b01031a0 0x7f13b019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 --2- 172.21.15.46:0/2132829675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b0101fb0 0x7f13b019a500 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.203+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 >> 172.21.15.46:0/2132829675 conn(0x7f13b00fd520 msgr2=0x7f13b0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.204+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 shutdown_connections 2024-04-17T08:01:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.204+0000 7f13b8000640 1 -- 172.21.15.46:0/2132829675 wait complete. 2024-04-17T08:01:38.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:37 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/704087326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:01:38.226 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:37 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/704087326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:01:38.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T08:01:38.237 INFO:tasks.workunit.client.0.smithi046.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-04-17T08:01:38.237 INFO:tasks.workunit.client.0.smithi046.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-17T08:01:38.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:37 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/704087326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 -- 172.21.15.46:0/3154579037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c103b70 msgr2=0x7f904c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 --2- 172.21.15.46:0/3154579037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c103b70 0x7f904c10a400 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9044009ca0 tx=0x7f904402c100 comp rx=0 tx=0).stop 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 -- 172.21.15.46:0/3154579037 shutdown_connections 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 --2- 172.21.15.46:0/3154579037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c103b70 0x7f904c10a400 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 --2- 172.21.15.46:0/3154579037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f904c1031b0 0x7f904c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 --2- 172.21.15.46:0/3154579037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904c101fc0 0x7f904c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 -- 172.21.15.46:0/3154579037 >> 172.21.15.46:0/3154579037 conn(0x7f904c0fd530 msgr2=0x7f904c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.431+0000 7f9051800640 1 -- 172.21.15.46:0/3154579037 shutdown_connections 2024-04-17T08:01:38.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.432+0000 7f9051800640 1 -- 172.21.15.46:0/3154579037 wait complete. 2024-04-17T08:01:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.433+0000 7f9051800640 1 Processor -- start 2024-04-17T08:01:38.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.433+0000 7f9051800640 1 -- start start 2024-04-17T08:01:38.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f9051800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904c101fc0 0x7f904c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f9051800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c1031b0 0x7f904c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f9051800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f904c103b70 0x7f904c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f9051800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f904c10dc50 con 0x7f904c101fc0 2024-04-17T08:01:38.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f9051800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f904c10dad0 con 0x7f904c103b70 2024-04-17T08:01:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f9051800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f904c10ddd0 con 0x7f904c1031b0 2024-04-17T08:01:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f904a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c1031b0 0x7f904c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f904aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904c101fc0 0x7f904c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f904a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c1031b0 0x7f904c19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58560/0 (socket says 172.21.15.46:58560) 2024-04-17T08:01:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.434+0000 7f904a000640 1 -- 172.21.15.46:0/1450188586 learned_addr learned my addr 172.21.15.46:0/1450188586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.435+0000 7f904b400640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f904c103b70 0x7f904c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.435+0000 7f904a000640 1 -- 172.21.15.46:0/1450188586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f904c103b70 msgr2=0x7f904c1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.435+0000 7f904a000640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f904c103b70 0x7f904c1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.435+0000 7f904a000640 1 -- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904c101fc0 msgr2=0x7f904c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.435+0000 7f904a000640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904c101fc0 0x7f904c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.435+0000 7f904a000640 1 -- 172.21.15.46:0/1450188586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c1a2090 con 0x7f904c1031b0 2024-04-17T08:01:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.435+0000 7f904a000640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c1031b0 0x7f904c19a9b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f903800d770 tx=0x7f903800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.436+0000 7f9043400640 1 -- 172.21.15.46:0/1450188586 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038017070 con 0x7f904c1031b0 2024-04-17T08:01:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.436+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c1a2380 con 0x7f904c1031b0 2024-04-17T08:01:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.436+0000 7f9043400640 1 -- 172.21.15.46:0/1450188586 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9038005500 con 0x7f904c1031b0 2024-04-17T08:01:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.436+0000 7f9043400640 1 -- 172.21.15.46:0/1450188586 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038010460 con 0x7f904c1031b0 2024-04-17T08:01:38.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.436+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c1a28d0 con 0x7f904c1031b0 2024-04-17T08:01:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.441+0000 7f905440b740 1 Processor -- start 2024-04-17T08:01:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.441+0000 7f905440b740 1 -- start start 2024-04-17T08:01:38.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.441+0000 7f905440b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453aaf63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.441+0000 7f905440b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453aafb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.441+0000 7f905440b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453aafdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.441+0000 7f905440b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56453aa5c020 con 0x56453aaf5fa0 2024-04-17T08:01:38.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f905440b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56453aa02460 con 0x56453aaf6990 2024-04-17T08:01:38.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f905440b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56453aa57fe0 con 0x56453aafbb00 2024-04-17T08:01:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f904b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453aaf63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f9043400640 1 -- 172.21.15.46:0/1450188586 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9038009c30 con 0x7f904c1031b0 2024-04-17T08:01:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f904b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453aaf63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54866/0 (socket says 172.21.15.46:54866) 2024-04-17T08:01:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f904b400640 1 -- 172.21.15.46:0/3438561218 learned_addr learned my addr 172.21.15.46:0/3438561218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f904aa00640 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453aafb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.442+0000 7f904a000640 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453aafdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f904aa00640 1 -- 172.21.15.46:0/3438561218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 msgr2=0x56453aafdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f904aa00640 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453aafdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f904aa00640 1 -- 172.21.15.46:0/3438561218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 msgr2=0x56453aaf63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f904aa00640 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453aaf63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f904aa00640 1 -- 172.21.15.46:0/3438561218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453aa0a040 con 0x56453aaf6990 2024-04-17T08:01:38.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f904aa00640 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453aafb490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f903c009bf0 tx=0x7f903c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1248efca9bff181 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f9037e00640 1 -- 172.21.15.46:0/3438561218 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c004050 con 0x56453aaf6990 2024-04-17T08:01:38.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f9037e00640 1 -- 172.21.15.46:0/3438561218 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903c005500 con 0x56453aaf6990 2024-04-17T08:01:38.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.443+0000 7f9037e00640 1 -- 172.21.15.46:0/3438561218 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c004e10 con 0x56453aaf6990 2024-04-17T08:01:38.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.444+0000 7f9043400640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90300a3bb0 0x7f90300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.444+0000 7f9043400640 1 -- 172.21.15.46:0/1450188586 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f90380d01b0 con 0x7f904c1031b0 2024-04-17T08:01:38.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.444+0000 7f904b400640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90300a3bb0 0x7f90300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.444+0000 7f905440b740 1 -- 172.21.15.46:0/3438561218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 msgr2=0x56453aafb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.444+0000 7f905440b740 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453aafb490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f903c009bf0 tx=0x7f903c011810 comp rx=0 tx=0).stop 2024-04-17T08:01:38.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f905440b740 1 -- 172.21.15.46:0/3438561218 shutdown_connections 2024-04-17T08:01:38.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f905440b740 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453aafdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f905440b740 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453aafb490 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f905440b740 1 --2- 172.21.15.46:0/3438561218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453aaf63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f905440b740 1 -- 172.21.15.46:0/3438561218 >> 172.21.15.46:0/3438561218 conn(0x56453aaf1510 msgr2=0x56453aaf1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f905440b740 1 -- 172.21.15.46:0/3438561218 shutdown_connections 2024-04-17T08:01:38.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f905440b740 1 -- 172.21.15.46:0/3438561218 wait complete. 2024-04-17T08:01:38.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.445+0000 7f904b400640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90300a3bb0 0x7f90300a6070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f9044032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.446+0000 7f905440b740 1 -- start start 2024-04-17T08:01:38.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.446+0000 7f905440b740 1 Processor -- start 2024-04-17T08:01:38.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.446+0000 7f905440b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453ab00510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.446+0000 7f905440b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab00a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.446+0000 7f905440b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab07a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.446+0000 7f905440b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56453aafeab0 con 0x56453aafbb00 2024-04-17T08:01:38.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f905440b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56453aa15830 con 0x56453aaf6990 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f905440b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56453aafec30 con 0x56453aaf5fa0 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f905440b740 10 client.0 _ll_register_callbacks cb 0x56453a9e3fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453ab00510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453ab00510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58576/0 (socket says 172.21.15.46:58576) 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 -- 172.21.15.46:0/2129905646 learned_addr learned my addr 172.21.15.46:0/2129905646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904b400640 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab07a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904a000640 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab00a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 -- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 msgr2=0x56453ab00a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab00a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 -- 172.21.15.46:0/2129905646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 msgr2=0x56453ab07a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab07a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.447+0000 7f904aa00640 1 -- 172.21.15.46:0/2129905646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453aa19920 con 0x56453aaf5fa0 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f904aa00640 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453ab00510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f903c01b040 tx=0x7f903c0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382b32a2d766c908 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f9034c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f9034c00640 1 -- 172.21.15.46:0/2129905646 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c029070 con 0x56453aaf5fa0 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f9034c00640 1 -- 172.21.15.46:0/2129905646 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f903c004d50 con 0x56453aaf5fa0 2024-04-17T08:01:38.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f903c005500 con 0x56453aaf5fa0 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f9034c00640 1 -- 172.21.15.46:0/2129905646 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f903c035050 con 0x56453aaf5fa0 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f905440b740 10 client.26170 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.448+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56453a9e77f0 con 0x56453aaf5fa0 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f9034c00640 1 -- 172.21.15.46:0/2129905646 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903c005500 con 0x56453aaf5fa0 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f9034c00640 1 -- 172.21.15.46:0/2129905646 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f903c023430 con 0x56453aaf5fa0 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f905440b740 10 client.26170 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f905440b740 10 client.26170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f905440b740 20 client.26170 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f905440b740 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab0bd80 0x56453ab0e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f905440b740 4 client.26170 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:38.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.449+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56453aa69a40 con 0x56453ab0bd80 2024-04-17T08:01:38.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.450+0000 7f904a000640 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab0bd80 0x56453ab0e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.450+0000 7f904a000640 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab0bd80 0x56453ab0e220 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957511111b9506c3 server_cookie=8d7e2b0c768233cb in_seq=0 out_seq=0 2024-04-17T08:01:38.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.450+0000 7f9034c00640 10 client.26170 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:38.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f9034c00640 1 -- 172.21.15.46:0/2129905646 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56453aa69a40 con 0x56453ab0bd80 2024-04-17T08:01:38.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f9034c00640 10 client.26170 handle_command_reply: tid=0 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 client.26170 shutdown 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 msgr2=0x56453ab00510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453ab00510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f903c01b040 tx=0x7f903c0045a0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 shutdown_connections 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab0bd80 0x56453ab0e220 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab07a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab00a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 --2- 172.21.15.46:0/2129905646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453ab00510 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 >> 172.21.15.46:0/2129905646 conn(0x56453aaf1510 msgr2=0x56453aaf0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.451+0000 7f9034c00640 0 client.26170 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:38.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.452+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 shutdown_connections 2024-04-17T08:01:38.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.452+0000 7f905440b740 1 -- 172.21.15.46:0/2129905646 wait complete. 2024-04-17T08:01:38.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.452+0000 7f905440b740 20 client.26170 trim_cache size 0 max 16384 2024-04-17T08:01:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 Processor -- start 2024-04-17T08:01:38.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 -- start start 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453aaffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab07a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab00070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1af640 con 0x56453aafbb00 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1acf00 con 0x56453aaf6990 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.467+0000 7f905440b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1aad50 con 0x56453aaf5fa0 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453aaffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453aaffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58578/0 (socket says 172.21.15.46:58578) 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 -- 172.21.15.46:0/4244486595 learned_addr learned my addr 172.21.15.46:0/4244486595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904a000640 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab07a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904b400640 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab00070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 -- 172.21.15.46:0/4244486595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 msgr2=0x56453ab07a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab07a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 -- 172.21.15.46:0/4244486595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 msgr2=0x56453ab00070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab00070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904aa00640 1 -- 172.21.15.46:0/4244486595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453a76e3f0 con 0x56453aaf5fa0 2024-04-17T08:01:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.468+0000 7f904a000640 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab07a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.469+0000 7f904aa00640 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453aaffb30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f903c001eb0 tx=0x7f903c009ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1703ad3b08b59d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.469+0000 7f9037400640 1 -- 172.21.15.46:0/4244486595 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c029050 con 0x56453aaf5fa0 2024-04-17T08:01:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.469+0000 7f9037400640 1 -- 172.21.15.46:0/4244486595 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903c023a40 con 0x56453aaf5fa0 2024-04-17T08:01:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.469+0000 7f9037400640 1 -- 172.21.15.46:0/4244486595 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c02e6a0 con 0x56453aaf5fa0 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.470+0000 7f905440b740 1 -- 172.21.15.46:0/4244486595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 msgr2=0x56453aaffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.470+0000 7f905440b740 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453aaffb30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f903c001eb0 tx=0x7f903c009ce0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.470+0000 7f905440b740 1 -- 172.21.15.46:0/4244486595 shutdown_connections 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.470+0000 7f905440b740 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aafbb00 0x56453ab00070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.470+0000 7f905440b740 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab07a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.470+0000 7f905440b740 1 --2- 172.21.15.46:0/4244486595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aaf5fa0 0x56453aaffb30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.470+0000 7f905440b740 1 -- 172.21.15.46:0/4244486595 >> 172.21.15.46:0/4244486595 conn(0x56453aaf1510 msgr2=0x56453aaf5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.471+0000 7f905440b740 1 -- 172.21.15.46:0/4244486595 shutdown_connections 2024-04-17T08:01:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.471+0000 7f905440b740 1 -- 172.21.15.46:0/4244486595 wait complete. 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.471+0000 7f905440b740 1 -- start start 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.471+0000 7f905440b740 1 Processor -- start 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f905440b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453ab04850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f905440b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab04d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f905440b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453ab70350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f905440b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1ab3c0 con 0x56453aaf5fa0 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f905440b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1ac830 con 0x56453aaf6990 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f905440b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1aba90 con 0x56453aafbb00 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f905440b740 10 client.0 _ll_register_callbacks cb 0x7f903c004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f904aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453ab04850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f904a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab04d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f904aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453ab04850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54902/0 (socket says 172.21.15.46:54902) 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f904aa00640 1 -- 172.21.15.46:0/2834819719 learned_addr learned my addr 172.21.15.46:0/2834819719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.472+0000 7f904b400640 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453ab70350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f904aa00640 1 -- 172.21.15.46:0/2834819719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 msgr2=0x56453ab70350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f904aa00640 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453ab70350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f904aa00640 1 -- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 msgr2=0x56453ab04d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f904aa00640 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab04d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f904aa00640 1 -- 172.21.15.46:0/2834819719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56453a8387d0 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f904a000640 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab04d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f904aa00640 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453ab04850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f903c0095f0 tx=0x7f903c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d30ae02f9382c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f9034c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f9034c00640 1 -- 172.21.15.46:0/2834819719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c03e070 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f903c023a40 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.473+0000 7f9034c00640 1 -- 172.21.15.46:0/2834819719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f903c0037f0 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.474+0000 7f9034c00640 1 -- 172.21.15.46:0/2834819719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f903c029040 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.474+0000 7f905440b740 10 client.16035 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.474+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56453ab71830 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.474+0000 7f9034c00640 1 -- 172.21.15.46:0/2834819719 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903c023a40 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.474+0000 7f9034c00640 1 -- 172.21.15.46:0/2834819719 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f903c041050 con 0x56453aaf5fa0 2024-04-17T08:01:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.475+0000 7f905440b740 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.475+0000 7f905440b740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.475+0000 7f905440b740 20 client.16035 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.475+0000 7f905440b740 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab720b0 0x56453ab744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.475+0000 7f905440b740 4 client.16035 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.475+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56453aa638b0 con 0x56453ab720b0 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.475+0000 7f904a000640 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab720b0 0x56453ab744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f9034c00640 10 client.16035 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f904a000640 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab720b0 0x56453ab744a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3a82fee3f3c0f0 server_cookie=a83aea4ac6ee92c9 in_seq=0 out_seq=0 2024-04-17T08:01:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f9034c00640 1 -- 172.21.15.46:0/2834819719 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56453aa638b0 con 0x56453ab720b0 2024-04-17T08:01:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f9034c00640 10 client.16035 handle_command_reply: tid=0 2024-04-17T08:01:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 client.16035 shutdown 2024-04-17T08:01:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 msgr2=0x56453ab04850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453ab04850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f903c0095f0 tx=0x7f903c009870 comp rx=0 tx=0).stop 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 shutdown_connections 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x56453ab720b0 0x56453ab744a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56453aafbb00 0x56453ab70350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56453aaf6990 0x56453ab04d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 --2- 172.21.15.46:0/2834819719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56453aaf5fa0 0x56453ab04850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 >> 172.21.15.46:0/2834819719 conn(0x56453aaf1510 msgr2=0x56453aaefe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.476+0000 7f9034c00640 0 client.16035 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.477+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 shutdown_connections 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.477+0000 7f905440b740 1 -- 172.21.15.46:0/2834819719 wait complete. 2024-04-17T08:01:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.477+0000 7f905440b740 20 client.16035 trim_cache size 0 max 16384 2024-04-17T08:01:38.479 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:01:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90300a3bb0 msgr2=0x7f90300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90300a3bb0 0x7f90300a6070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f9044032590 comp rx=0 tx=0).stop 2024-04-17T08:01:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c1031b0 msgr2=0x7f904c19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c1031b0 0x7f904c19a9b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f903800d770 tx=0x7f903800dc30 comp rx=0 tx=0).stop 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 shutdown_connections 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f90300a3bb0 0x7f90300a6070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f904c103b70 0x7f904c1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f904c1031b0 0x7f904c19a9b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 --2- 172.21.15.46:0/1450188586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f904c101fc0 0x7f904c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 >> 172.21.15.46:0/1450188586 conn(0x7f904c0fd530 msgr2=0x7f904c074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 shutdown_connections 2024-04-17T08:01:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:38.478+0000 7f9051800640 1 -- 172.21.15.46:0/1450188586 wait complete. 2024-04-17T08:01:38.658 INFO:tasks.workunit.client.0.smithi046.stderr:doio.c: In function 'parse_cmdline': 2024-04-17T08:01:38.658 INFO:tasks.workunit.client.0.smithi046.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-04-17T08:01:38.658 INFO:tasks.workunit.client.0.smithi046.stderr: 3113 | fprintf(stderr, 2024-04-17T08:01:38.658 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~ 2024-04-17T08:01:38.658 INFO:tasks.workunit.client.0.smithi046.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-04-17T08:01:38.658 INFO:tasks.workunit.client.0.smithi046.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-17T08:01:38.659 INFO:tasks.workunit.client.0.smithi046.stderr: 3115 | Prog, TagName, tok); 2024-04-17T08:01:38.659 INFO:tasks.workunit.client.0.smithi046.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-04-17T08:01:38.709 INFO:tasks.workunit.client.0.smithi046.stderr:fsx.c: In function 'main': 2024-04-17T08:01:38.709 INFO:tasks.workunit.client.0.smithi046.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-04-17T08:01:38.709 INFO:tasks.workunit.client.0.smithi046.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-04-17T08:01:38.709 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-17T08:01:38.709 INFO:tasks.workunit.client.0.smithi046.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-04-17T08:01:38.709 INFO:tasks.workunit.client.0.smithi046.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-04-17T08:01:38.709 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-17T08:01:38.858 INFO:tasks.workunit.client.0.smithi046.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-04-17T08:01:38.858 INFO:tasks.workunit.client.0.smithi046.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-04-17T08:01:39.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:38 smithi053 ceph-mon[36437]: pgmap v511: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 758 KiB/s wr, 89 op/s 2024-04-17T08:01:39.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:38 smithi062 ceph-mon[36507]: pgmap v511: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 758 KiB/s wr, 89 op/s 2024-04-17T08:01:39.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:38 smithi046 ceph-mon[32038]: pgmap v511: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 758 KiB/s wr, 89 op/s 2024-04-17T08:01:39.496 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T08:01:39.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:01:39.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 -- 172.21.15.46:0/1410028831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 msgr2=0x7fd660105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 --2- 172.21.15.46:0/1410028831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 0x7fd660105820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009900 tx=0x7fd64c02bc30 comp rx=0 tx=0).stop 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 -- 172.21.15.46:0/1410028831 shutdown_connections 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 --2- 172.21.15.46:0/1410028831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd660105d60 0x7fd66010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 --2- 172.21.15.46:0/1410028831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 0x7fd660105820 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 --2- 172.21.15.46:0/1410028831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 0x7fd6601045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 -- 172.21.15.46:0/1410028831 >> 172.21.15.46:0/1410028831 conn(0x7fd6600ff720 msgr2=0x7fd660101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 -- 172.21.15.46:0/1410028831 shutdown_connections 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.733+0000 7fd665c00640 1 -- 172.21.15.46:0/1410028831 wait complete. 2024-04-17T08:01:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.734+0000 7fd665c00640 1 Processor -- start 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.734+0000 7fd665c00640 1 -- start start 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd665c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 0x7fd66019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd665c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 0x7fd66019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd665c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd660105d60 0x7fd6601a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd665c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd66010fe40 con 0x7fd660105d60 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd665c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd66010fcc0 con 0x7fd6601041b0 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd665c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd66010ffc0 con 0x7fd6601053a0 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd65ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 0x7fd66019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd65ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 0x7fd66019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46814/0 (socket says 172.21.15.46:46814) 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd65ea00640 1 -- 172.21.15.46:0/138773978 learned_addr learned my addr 172.21.15.46:0/138773978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd65f400640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd660105d60 0x7fd6601a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.735+0000 7fd65e000640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 0x7fd66019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd65ea00640 1 -- 172.21.15.46:0/138773978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 msgr2=0x7fd66019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd65ea00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 0x7fd66019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd65ea00640 1 -- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd660105d60 msgr2=0x7fd6601a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd65ea00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd660105d60 0x7fd6601a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd65ea00640 1 -- 172.21.15.46:0/138773978 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6601a2110 con 0x7fd6601041b0 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd65ea00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 0x7fd66019a4f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd65000e9c0 tx=0x7fd65000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd657400640 1 -- 172.21.15.46:0/138773978 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65000cd80 con 0x7fd6601041b0 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6601a2400 con 0x7fd6601041b0 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd657400640 1 -- 172.21.15.46:0/138773978 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6500035f0 con 0x7fd6601041b0 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd657400640 1 -- 172.21.15.46:0/138773978 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650010690 con 0x7fd6601041b0 2024-04-17T08:01:39.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.736+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601a2920 con 0x7fd6601041b0 2024-04-17T08:01:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 Processor -- start 2024-04-17T08:01:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 -- start start 2024-04-17T08:01:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd657400640 1 -- 172.21.15.46:0/138773978 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd6500037a0 con 0x7fd6601041b0 2024-04-17T08:01:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7630ec0 0x5609c76312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c76363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7636a20 0x5609c7638eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609c7596cb0 con 0x5609c7630ec0 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5609c753d370 con 0x5609c7636a20 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd66886d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5609c7592c70 con 0x5609c76318b0 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.740+0000 7fd65f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7630ec0 0x5609c76312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7636a20 0x5609c7638eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7630ec0 0x5609c76312e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54916/0 (socket says 172.21.15.46:54916) 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65f400640 1 -- 172.21.15.46:0/1221271949 learned_addr learned my addr 172.21.15.46:0/1221271949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c76363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65e000640 1 -- 172.21.15.46:0/1221271949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 msgr2=0x5609c76363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65e000640 1 --2- 172.21.15.46:0/1221271949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c76363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65e000640 1 -- 172.21.15.46:0/1221271949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7630ec0 msgr2=0x5609c76312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65e000640 1 --2- 172.21.15.46:0/1221271949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7630ec0 0x5609c76312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65e000640 1 -- 172.21.15.46:0/1221271949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c755c910 con 0x5609c7636a20 2024-04-17T08:01:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.741+0000 7fd65e000640 1 --2- 172.21.15.46:0/1221271949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7636a20 0x5609c7638eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009230 tx=0x7fd64c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2741bbc9fc4d1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.742+0000 7fd64be00640 1 -- 172.21.15.46:0/1221271949 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c02cb80 con 0x5609c7636a20 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.742+0000 7fd657400640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6440a3c30 0x7fd6440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.742+0000 7fd64be00640 1 -- 172.21.15.46:0/1221271949 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64c005040 con 0x5609c7636a20 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.742+0000 7fd64be00640 1 -- 172.21.15.46:0/1221271949 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c004900 con 0x5609c7636a20 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.742+0000 7fd657400640 1 -- 172.21.15.46:0/138773978 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd6500d0ad0 con 0x7fd6601041b0 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.742+0000 7fd65f400640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6440a3c30 0x7fd6440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 -- 172.21.15.46:0/1221271949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7636a20 msgr2=0x5609c7638eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 --2- 172.21.15.46:0/1221271949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7636a20 0x5609c7638eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009230 tx=0x7fd64c009560 comp rx=0 tx=0).stop 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd65f400640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6440a3c30 0x7fd6440a60f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5609c7632110 tx=0x7fd658009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 -- 172.21.15.46:0/1221271949 shutdown_connections 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 --2- 172.21.15.46:0/1221271949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7636a20 0x5609c7638eb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 --2- 172.21.15.46:0/1221271949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c76363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 --2- 172.21.15.46:0/1221271949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7630ec0 0x5609c76312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 -- 172.21.15.46:0/1221271949 >> 172.21.15.46:0/1221271949 conn(0x5609c762c430 msgr2=0x5609c762c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 -- 172.21.15.46:0/1221271949 shutdown_connections 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.743+0000 7fd66886d740 1 -- 172.21.15.46:0/1221271949 wait complete. 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 -- start start 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 Processor -- start 2024-04-17T08:01:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c763b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c763b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7642900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609c7639970 con 0x5609c7636a20 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5609c7550730 con 0x5609c7630ec0 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5609c7639af0 con 0x5609c76318b0 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.744+0000 7fd66886d740 10 client.0 _ll_register_callbacks cb 0x5609c7552b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c763b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7642900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c763b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c763b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58624/0 (socket says 172.21.15.46:58624) 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 -- 172.21.15.46:0/381868452 learned_addr learned my addr 172.21.15.46:0/381868452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 -- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 msgr2=0x5609c763b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c763b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 -- 172.21.15.46:0/381868452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 msgr2=0x5609c7642900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7642900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 -- 172.21.15.46:0/381868452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c73717e0 con 0x5609c76318b0 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65ea00640 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c763b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd65e000640 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c763b8c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02c170 tx=0x7fd64c004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8575460916bf2e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.745+0000 7fd648c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.746+0000 7fd648c00640 1 -- 172.21.15.46:0/381868452 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c005400 con 0x5609c76318b0 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.746+0000 7fd648c00640 1 -- 172.21.15.46:0/381868452 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd64c0055a0 con 0x5609c76318b0 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.746+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd64c005040 con 0x5609c76318b0 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.746+0000 7fd648c00640 1 -- 172.21.15.46:0/381868452 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd64c0046c0 con 0x5609c76318b0 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.746+0000 7fd66886d740 10 client.26188 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.746+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609c73d1f70 con 0x5609c76318b0 2024-04-17T08:01:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.746+0000 7fd648c00640 1 -- 172.21.15.46:0/381868452 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd64c005040 con 0x5609c76318b0 2024-04-17T08:01:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd648c00640 1 -- 172.21.15.46:0/381868452 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fd64c003a90 con 0x5609c76318b0 2024-04-17T08:01:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd66886d740 10 client.26188 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd66886d740 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd66886d740 20 client.26188 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd66886d740 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c7646df0 0x5609c76491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd66886d740 4 client.26188 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c75a46d0 con 0x5609c7646df0 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.747+0000 7fd65ea00640 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c7646df0 0x5609c76491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.748+0000 7fd65ea00640 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c7646df0 0x5609c76491e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95c768ae5663081 server_cookie=5c9871aac63cc202 in_seq=0 out_seq=0 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.748+0000 7fd648c00640 10 client.26188 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd648c00640 1 -- 172.21.15.46:0/381868452 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609c75a46d0 con 0x5609c7646df0 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd648c00640 10 client.26188 handle_command_reply: tid=0 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 client.26188 shutdown 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 msgr2=0x5609c763b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c763b8c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02c170 tx=0x7fd64c004260 comp rx=0 tx=0).stop 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 shutdown_connections 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c7646df0 0x5609c76491e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7642900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c763b8c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 --2- 172.21.15.46:0/381868452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c763b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd648c00640 0 client.26188 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 >> 172.21.15.46:0/381868452 conn(0x5609c762c430 msgr2=0x5609c762b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 shutdown_connections 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 1 -- 172.21.15.46:0/381868452 wait complete. 2024-04-17T08:01:39.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.749+0000 7fd66886d740 20 client.26188 trim_cache size 0 max 16384 2024-04-17T08:01:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 Processor -- start 2024-04-17T08:01:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 -- start start 2024-04-17T08:01:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c762b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c76318b0 0x5609c76428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c7636a20 0x5609c764da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601ab1f0 con 0x5609c76318b0 2024-04-17T08:01:39.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601a8ab0 con 0x5609c7630ec0 2024-04-17T08:01:39.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.769+0000 7fd66886d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601a6920 con 0x5609c7636a20 2024-04-17T08:01:39.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c76318b0 0x5609c76428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c762b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c762b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46836/0 (socket says 172.21.15.46:46836) 2024-04-17T08:01:39.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 -- 172.21.15.46:0/4196125623 learned_addr learned my addr 172.21.15.46:0/4196125623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:39.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65f400640 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c7636a20 0x5609c764da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 -- 172.21.15.46:0/4196125623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c7636a20 msgr2=0x5609c764da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c7636a20 0x5609c764da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 -- 172.21.15.46:0/4196125623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c76318b0 msgr2=0x5609c76428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c76318b0 0x5609c76428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.770+0000 7fd65ea00640 1 -- 172.21.15.46:0/4196125623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c7269170 con 0x5609c7630ec0 2024-04-17T08:01:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.771+0000 7fd65ea00640 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c762b460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd650098e20 tx=0x7fd6500bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21228ccf420b461f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.771+0000 7fd64b400640 1 -- 172.21.15.46:0/4196125623 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65009c790 con 0x5609c7630ec0 2024-04-17T08:01:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.771+0000 7fd64b400640 1 -- 172.21.15.46:0/4196125623 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65009d0c0 con 0x5609c7630ec0 2024-04-17T08:01:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.771+0000 7fd64b400640 1 -- 172.21.15.46:0/4196125623 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65009c320 con 0x5609c7630ec0 2024-04-17T08:01:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 -- 172.21.15.46:0/4196125623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 msgr2=0x5609c762b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c762b460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd650098e20 tx=0x7fd6500bd040 comp rx=0 tx=0).stop 2024-04-17T08:01:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 -- 172.21.15.46:0/4196125623 shutdown_connections 2024-04-17T08:01:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c7636a20 0x5609c764da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c76318b0 0x5609c76428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 --2- 172.21.15.46:0/4196125623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c762b460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 -- 172.21.15.46:0/4196125623 >> 172.21.15.46:0/4196125623 conn(0x5609c762c430 msgr2=0x5609c762ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 -- 172.21.15.46:0/4196125623 shutdown_connections 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.772+0000 7fd66886d740 1 -- 172.21.15.46:0/4196125623 wait complete. 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.773+0000 7fd66886d740 1 -- start start 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.773+0000 7fd66886d740 1 Processor -- start 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.773+0000 7fd66886d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c76407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.773+0000 7fd66886d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c7640d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd66886d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7641d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd66886d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601a6f70 con 0x5609c7636a20 2024-04-17T08:01:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd66886d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601a83e0 con 0x5609c7630ec0 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd66886d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6601a7640 con 0x5609c76318b0 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd66886d740 10 client.0 _ll_register_callbacks cb 0x7fd658009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c7640d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c7640d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58638/0 (socket says 172.21.15.46:58638) 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 -- 172.21.15.46:0/2824645890 learned_addr learned my addr 172.21.15.46:0/2824645890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65ea00640 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c76407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65f400640 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7641d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 -- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 msgr2=0x5609c76407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c76407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 -- 172.21.15.46:0/2824645890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 msgr2=0x5609c7641d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7641d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.774+0000 7fd65e000640 1 -- 172.21.15.46:0/2824645890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c7642300 con 0x5609c76318b0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd65e000640 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c7640d00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02e040 tx=0x7fd64c004500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11379c046eeb441 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd648c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd648c00640 1 -- 172.21.15.46:0/2824645890 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c005610 con 0x5609c76318b0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd648c00640 1 -- 172.21.15.46:0/2824645890 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd64c034880 con 0x5609c76318b0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd65009d0c0 con 0x5609c76318b0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd648c00640 1 -- 172.21.15.46:0/2824645890 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd64c034a80 con 0x5609c76318b0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd66886d740 10 client.26194 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.775+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609c76adb00 con 0x5609c76318b0 2024-04-17T08:01:39.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd648c00640 1 -- 172.21.15.46:0/2824645890 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65009d0c0 con 0x5609c76318b0 2024-04-17T08:01:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd648c00640 1 -- 172.21.15.46:0/2824645890 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7fd64c004860 con 0x5609c76318b0 2024-04-17T08:01:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd66886d740 10 client.26194 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd66886d740 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd66886d740 20 client.26194 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd66886d740 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c76ae440 0x5609c76b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd66886d740 4 client.26194 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5609c759e540 con 0x5609c76ae440 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.776+0000 7fd65ea00640 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c76ae440 0x5609c76b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.777+0000 7fd65ea00640 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c76ae440 0x5609c76b0860 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c98761b8884c38 server_cookie=523acbe0db546cfc in_seq=0 out_seq=0 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.777+0000 7fd648c00640 10 client.26194 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.778+0000 7fd648c00640 1 -- 172.21.15.46:0/2824645890 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5609c759e540 con 0x5609c76ae440 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.778+0000 7fd648c00640 10 client.26194 handle_command_reply: tid=0 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.778+0000 7fd66886d740 1 client.26194 shutdown 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.778+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 msgr2=0x5609c7640d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.778+0000 7fd66886d740 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c7640d00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd64c02e040 tx=0x7fd64c004500 comp rx=0 tx=0).stop 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 shutdown_connections 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5609c76ae440 0x5609c76b0860 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609c7636a20 0x5609c7641d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609c76318b0 0x5609c7640d00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 --2- 172.21.15.46:0/2824645890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609c7630ec0 0x5609c76407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 >> 172.21.15.46:0/2824645890 conn(0x5609c762c430 msgr2=0x5609c762b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd648c00640 0 client.26194 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 shutdown_connections 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 1 -- 172.21.15.46:0/2824645890 wait complete. 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.779+0000 7fd66886d740 20 client.26194 trim_cache size 0 max 16384 2024-04-17T08:01:39.781 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:01:39.782 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-17T08:01:39.782 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "97f91266-620b-4a89-aa0a-9798c84dbe40", 2024-04-17T08:01:39.782 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-17T08:01:39.782 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6440a3c30 msgr2=0x7fd6440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6440a3c30 0x7fd6440a60f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x5609c7632110 tx=0x7fd658009500 comp rx=0 tx=0).stop 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 msgr2=0x7fd66019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 0x7fd66019a4f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd65000e9c0 tx=0x7fd65000ee80 comp rx=0 tx=0).stop 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 shutdown_connections 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fd6440a3c30 0x7fd6440a60f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd660105d60 0x7fd6601a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6601053a0 0x7fd66019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 --2- 172.21.15.46:0/138773978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6601041b0 0x7fd66019a4f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 >> 172.21.15.46:0/138773978 conn(0x7fd6600ff720 msgr2=0x7fd6601086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:39.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 shutdown_connections 2024-04-17T08:01:39.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:39.780+0000 7fd665c00640 1 -- 172.21.15.46:0/138773978 wait complete. 2024-04-17T08:01:39.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:01:39.798 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:01:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 -- 172.21.15.46:0/4279429604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880c1053a0 msgr2=0x7f880c105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 --2- 172.21.15.46:0/4279429604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880c1053a0 0x7f880c105820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802bc30 comp rx=0 tx=0).stop 2024-04-17T08:01:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 -- 172.21.15.46:0/4279429604 shutdown_connections 2024-04-17T08:01:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 --2- 172.21.15.46:0/4279429604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f880c105d60 0x7f880c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 --2- 172.21.15.46:0/4279429604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880c1053a0 0x7f880c105820 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 --2- 172.21.15.46:0/4279429604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 0x7f880c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 -- 172.21.15.46:0/4279429604 >> 172.21.15.46:0/4279429604 conn(0x7f880c0ff720 msgr2=0x7f880c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 -- 172.21.15.46:0/4279429604 shutdown_connections 2024-04-17T08:01:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.079+0000 7f8812c00640 1 -- 172.21.15.46:0/4279429604 wait complete. 2024-04-17T08:01:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 Processor -- start 2024-04-17T08:01:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 -- start start 2024-04-17T08:01:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 0x7f880c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f880c1053a0 0x7f880c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880c105d60 0x7f880c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10fe90 con 0x7f880c1053a0 2024-04-17T08:01:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10fd10 con 0x7f880c105d60 2024-04-17T08:01:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f8812c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f880c110010 con 0x7f880c1041b0 2024-04-17T08:01:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.081+0000 7f880be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 0x7f880c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 0x7f880c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:58658/0 (socket says 172.21.15.46:58658) 2024-04-17T08:01:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 -- 172.21.15.46:0/1447660801 learned_addr learned my addr 172.21.15.46:0/1447660801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880b400640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f880c1053a0 0x7f880c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 -- 172.21.15.46:0/1447660801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880c105d60 msgr2=0x7f880c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880c105d60 0x7f880c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 -- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f880c1053a0 msgr2=0x7f880c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f880c1053a0 0x7f880c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 -- 172.21.15.46:0/1447660801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c1a20d0 con 0x7f880c1041b0 2024-04-17T08:01:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880b400640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f880c1053a0 0x7f880c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.082+0000 7f880be00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 0x7f880c19a4b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e9c0 tx=0x7f87fc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.083+0000 7f8808c00640 1 -- 172.21.15.46:0/1447660801 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00cd80 con 0x7f880c1041b0 2024-04-17T08:01:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.083+0000 7f8808c00640 1 -- 172.21.15.46:0/1447660801 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc0035f0 con 0x7f880c1041b0 2024-04-17T08:01:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.083+0000 7f8808c00640 1 -- 172.21.15.46:0/1447660801 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc010690 con 0x7f880c1041b0 2024-04-17T08:01:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.083+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1a23c0 con 0x7f880c1041b0 2024-04-17T08:01:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.083+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1a2860 con 0x7f880c1041b0 2024-04-17T08:01:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.089+0000 7f88159e8740 1 Processor -- start 2024-04-17T08:01:40.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.089+0000 7f8808c00640 1 -- 172.21.15.46:0/1447660801 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f87fc0037a0 con 0x7f880c1041b0 2024-04-17T08:01:40.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.089+0000 7f88159e8740 1 -- start start 2024-04-17T08:01:40.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.090+0000 7f88159e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3190 0x55ee3a8d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.090+0000 7f88159e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.090+0000 7f88159e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.090+0000 7f88159e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ee3a83a120 con 0x55ee3a8d3b80 2024-04-17T08:01:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.090+0000 7f88159e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ee3a7e0340 con 0x55ee3a8d3190 2024-04-17T08:01:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.090+0000 7f88159e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ee3a8360e0 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.090+0000 7f8810c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3190 0x55ee3a8d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.091+0000 7f8810c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3190 0x55ee3a8d35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46868/0 (socket says 172.21.15.46:46868) 2024-04-17T08:01:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.091+0000 7f8810c00640 1 -- 172.21.15.46:0/1446804414 learned_addr learned my addr 172.21.15.46:0/1446804414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.091+0000 7f880b400640 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.091+0000 7f8810c00640 1 -- 172.21.15.46:0/1446804414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d8cf0 msgr2=0x55ee3a8db180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.091+0000 7f8810c00640 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8db180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.091+0000 7f8810c00640 1 -- 172.21.15.46:0/1446804414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 msgr2=0x55ee3a8d8680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T08:01:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.092+0000 7f8808c00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f87f00a3cb0 0x7f87f00a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.092+0000 7f8808c00640 1 -- 172.21.15.46:0/1447660801 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f87fc0d0c10 con 0x7f880c1041b0 2024-04-17T08:01:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.093+0000 7f880be00640 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.093+0000 7f8810c00640 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8d8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.093+0000 7f8810c00640 1 -- 172.21.15.46:0/1446804414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3a6167d0 con 0x55ee3a8d3190 2024-04-17T08:01:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.093+0000 7f880be00640 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T08:01:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.094+0000 7f8810c00640 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3190 0x55ee3a8d35b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ea0 tx=0x7f8804009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9212509798b5c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.094+0000 7f880b400640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f87f00a3cb0 0x7f87f00a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.094+0000 7f8801200640 1 -- 172.21.15.46:0/1446804414 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004d20 con 0x55ee3a8d3190 2024-04-17T08:01:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.094+0000 7f8801200640 1 -- 172.21.15.46:0/1446804414 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f880400ee60 con 0x55ee3a8d3190 2024-04-17T08:01:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.094+0000 7f8801200640 1 -- 172.21.15.46:0/1446804414 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88040108a0 con 0x55ee3a8d3190 2024-04-17T08:01:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 -- 172.21.15.46:0/1446804414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3190 msgr2=0x55ee3a8d35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3190 0x55ee3a8d35b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ea0 tx=0x7f8804009f90 comp rx=0 tx=0).stop 2024-04-17T08:01:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 -- 172.21.15.46:0/1446804414 shutdown_connections 2024-04-17T08:01:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8db180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 --2- 172.21.15.46:0/1446804414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3190 0x55ee3a8d35b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 -- 172.21.15.46:0/1446804414 >> 172.21.15.46:0/1446804414 conn(0x55ee3a8ce700 msgr2=0x55ee3a8ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f880b400640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f87f00a3cb0 0x7f87f00a6170 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006010 tx=0x7f87f802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 -- 172.21.15.46:0/1446804414 shutdown_connections 2024-04-17T08:01:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.096+0000 7f88159e8740 1 -- 172.21.15.46:0/1446804414 wait complete. 2024-04-17T08:01:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.097+0000 7f88159e8740 1 -- start start 2024-04-17T08:01:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.097+0000 7f88159e8740 1 Processor -- start 2024-04-17T08:01:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f88159e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f88159e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f88159e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f880b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f88159e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ee3a8eccb0 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f88159e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ee3a7f3710 con 0x55ee3a8d3b80 2024-04-17T08:01:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f88159e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ee3a8ece30 con 0x55ee3a8d3190 2024-04-17T08:01:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f88159e8740 10 client.0 _ll_register_callbacks cb 0x55ee3a7eeaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f880b400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46870/0 (socket says 172.21.15.46:46870) 2024-04-17T08:01:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f880b400640 1 -- 172.21.15.46:0/1370225405 learned_addr learned my addr 172.21.15.46:0/1370225405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.098+0000 7f880be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f880be00640 1 -- 172.21.15.46:0/1370225405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3b80 msgr2=0x55ee3a8eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f880be00640 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8eecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f880be00640 1 -- 172.21.15.46:0/1370225405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d8cf0 msgr2=0x55ee3a8f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f880be00640 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.099+0000 7f880be00640 1 -- 172.21.15.46:0/1370225405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3a54c3f0 con 0x55ee3a8d3190 2024-04-17T08:01:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f880be00640 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ce320 tx=0x7f87fc09d2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436f0e0652486b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f87f5600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-17T08:01:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f87f5600640 1 -- 172.21.15.46:0/1370225405 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0b6060 con 0x55ee3a8d3190 2024-04-17T08:01:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f87f5600640 1 -- 172.21.15.46:0/1370225405 <== mon.1 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f87fc003b10 con 0x55ee3a8d3190 2024-04-17T08:01:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f880400ee60 con 0x55ee3a8d3190 2024-04-17T08:01:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f87f5600640 1 -- 172.21.15.46:0/1370225405 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f87fc0a5770 con 0x55ee3a8d3190 2024-04-17T08:01:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f88159e8740 10 client.26206 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee3a50c0f0 con 0x55ee3a8d3190 2024-04-17T08:01:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.100+0000 7f87f5600640 1 -- 172.21.15.46:0/1370225405 <== mon.1 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f880400ee60 con 0x55ee3a8d3190 2024-04-17T08:01:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.101+0000 7f87f5600640 1 -- 172.21.15.46:0/1370225405 <== mon.1 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f87fc09bf50 con 0x55ee3a8d3190 2024-04-17T08:01:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.101+0000 7f88159e8740 10 client.26206 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.101+0000 7f88159e8740 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.101+0000 7f88159e8740 20 client.26206 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.101+0000 7f88159e8740 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a8fa650 0x55ee3a8fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.101+0000 7f88159e8740 4 client.26206 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:01:40.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.101+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee3a847b40 con 0x55ee3a8fa650 2024-04-17T08:01:40.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.102+0000 7f880b400640 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a8fa650 0x55ee3a8fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.102+0000 7f880b400640 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a8fa650 0x55ee3a8fcad0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81dac69dfce2946a server_cookie=923a6f6b2bea5777 in_seq=0 out_seq=0 2024-04-17T08:01:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.102+0000 7f87f5600640 10 client.26206 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.103+0000 7f87f5600640 1 -- 172.21.15.46:0/1370225405 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee3a847b40 con 0x55ee3a8fa650 2024-04-17T08:01:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.103+0000 7f87f5600640 10 client.26206 handle_command_reply: tid=0 2024-04-17T08:01:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.103+0000 7f88159e8740 1 client.26206 shutdown 2024-04-17T08:01:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.103+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 msgr2=0x55ee3a8ee780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.103+0000 7f88159e8740 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ce320 tx=0x7f87fc09d2c0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 shutdown_connections 2024-04-17T08:01:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a8fa650 0x55ee3a8fcad0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 --2- 172.21.15.46:0/1370225405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee780 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 >> 172.21.15.46:0/1370225405 conn(0x55ee3a8ce700 msgr2=0x55ee3a8cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f87f5600640 0 client.26206 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 shutdown_connections 2024-04-17T08:01:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 1 -- 172.21.15.46:0/1370225405 wait complete. 2024-04-17T08:01:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.104+0000 7f88159e8740 20 client.26206 trim_cache size 0 max 16384 2024-04-17T08:01:40.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.141+0000 7f88159e8740 1 Processor -- start 2024-04-17T08:01:40.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.141+0000 7f88159e8740 1 -- start start 2024-04-17T08:01:40.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f88159e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f88159e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f88159e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f88159e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a6ef0 con 0x55ee3a8d3190 2024-04-17T08:01:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f88159e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a47b0 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f88159e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f880c073c10 con 0x55ee3a8d3b80 2024-04-17T08:01:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f8810c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f8810c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8ee8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46880/0 (socket says 172.21.15.46:46880) 2024-04-17T08:01:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f8810c00640 1 -- 172.21.15.46:0/99579646 learned_addr learned my addr 172.21.15.46:0/99579646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.142+0000 7f880b400640 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.143+0000 7f880be00640 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.143+0000 7f8810c00640 1 -- 172.21.15.46:0/99579646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3b80 msgr2=0x55ee3a8f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.143+0000 7f8810c00640 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.143+0000 7f8810c00640 1 -- 172.21.15.46:0/99579646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3190 msgr2=0x55ee3a8ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.143+0000 7f8810c00640 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.143+0000 7f8810c00640 1 -- 172.21.15.46:0/99579646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3a8d4410 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.143+0000 7f8810c00640 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8ee8d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8804005c70 tx=0x7f8804004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4f14584e454404d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.144+0000 7f87f7e00640 1 -- 172.21.15.46:0/99579646 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804026020 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.144+0000 7f87f7e00640 1 -- 172.21.15.46:0/99579646 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804009b60 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.144+0000 7f87f7e00640 1 -- 172.21.15.46:0/99579646 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880401d040 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 -- 172.21.15.46:0/99579646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 msgr2=0x55ee3a8ee8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8ee8d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8804005c70 tx=0x7f8804004980 comp rx=0 tx=0).stop 2024-04-17T08:01:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 -- 172.21.15.46:0/99579646 shutdown_connections 2024-04-17T08:01:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8ee8d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 --2- 172.21.15.46:0/99579646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3190 0x55ee3a8ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 -- 172.21.15.46:0/99579646 >> 172.21.15.46:0/99579646 conn(0x55ee3a8ce700 msgr2=0x55ee3a8ed1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 -- 172.21.15.46:0/99579646 shutdown_connections 2024-04-17T08:01:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.145+0000 7f88159e8740 1 -- 172.21.15.46:0/99579646 wait complete. 2024-04-17T08:01:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.146+0000 7f88159e8740 1 -- start start 2024-04-17T08:01:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.146+0000 7f88159e8740 1 Processor -- start 2024-04-17T08:01:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f88159e8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f88159e8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f88159e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f88159e8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a2c70 con 0x55ee3a8d3b80 2024-04-17T08:01:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f88159e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a40e0 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f88159e8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1a3340 con 0x55ee3a8d3190 2024-04-17T08:01:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f88159e8740 10 client.0 _ll_register_callbacks cb 0x7f87fc003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:01:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f8810c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f8810c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:46894/0 (socket says 172.21.15.46:46894) 2024-04-17T08:01:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f8810c00640 1 -- 172.21.15.46:0/229773527 learned_addr learned my addr 172.21.15.46:0/229773527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:01:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.147+0000 7f880b400640 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.148+0000 7f880be00640 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.148+0000 7f8810c00640 1 -- 172.21.15.46:0/229773527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 msgr2=0x55ee3a8f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:40.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.148+0000 7f8810c00640 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.148+0000 7f8810c00640 1 -- 172.21.15.46:0/229773527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 msgr2=0x55ee3a8f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:01:40.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.148+0000 7f8810c00640 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.148+0000 7f8810c00640 1 -- 172.21.15.46:0/229773527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3a8f4d60 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.149+0000 7f8810c00640 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f4770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8804009560 tx=0x7f88040052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d86d5a7e8511747 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:01:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.149+0000 7f87f5600640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:01:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.149+0000 7f87f5600640 1 -- 172.21.15.46:0/229773527 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804030020 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.149+0000 7f87f5600640 1 -- 172.21.15.46:0/229773527 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f880402b480 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.149+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8804009b60 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.150+0000 7f87f5600640 1 -- 172.21.15.46:0/229773527 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f880403b050 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.150+0000 7f88159e8740 10 client.26231 fetch_fsmap learned FSMap version 26 2024-04-17T08:01:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.150+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee3a969420 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.150+0000 7f87f5600640 1 -- 172.21.15.46:0/229773527 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804009b60 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.150+0000 7f87f5600640 1 -- 172.21.15.46:0/229773527 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 26) ==== 8355+0+0 (secure 0 0 0) 0x7f8804010530 con 0x55ee3a8d8cf0 2024-04-17T08:01:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.150+0000 7f88159e8740 10 client.26231 fetch_fsmap finished waiting for FSMap version 26 2024-04-17T08:01:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.151+0000 7f88159e8740 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:01:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.151+0000 7f88159e8740 20 client.26231 populate_metadata read hostname 'smithi046' 2024-04-17T08:01:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.151+0000 7f88159e8740 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a969a80 0x55ee3a96bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:01:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.151+0000 7f88159e8740 4 client.26231 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:01:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.151+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee3a8419b0 con 0x55ee3a969a80 2024-04-17T08:01:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.151+0000 7f880be00640 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a969a80 0x55ee3a96bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:01:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.152+0000 7f880be00640 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a969a80 0x55ee3a96bea0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a9adc6e2fa1ae1 server_cookie=3ce1db3e2c7b1d51 in_seq=0 out_seq=0 2024-04-17T08:01:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.152+0000 7f87f5600640 10 client.26231 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.152+0000 7f87f5600640 1 -- 172.21.15.46:0/229773527 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ee3a8419b0 con 0x55ee3a969a80 2024-04-17T08:01:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.152+0000 7f87f5600640 10 client.26231 handle_command_reply: tid=0 2024-04-17T08:01:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.152+0000 7f88159e8740 1 client.26231 shutdown 2024-04-17T08:01:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 msgr2=0x55ee3a8f4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f4770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8804009560 tx=0x7f88040052d0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 shutdown_connections 2024-04-17T08:01:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x55ee3a969a80 0x55ee3a96bea0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ee3a8d8cf0 0x55ee3a8f4770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee3a8d3b80 0x55ee3a8f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 --2- 172.21.15.46:0/229773527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee3a8d3190 0x55ee3a8f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 >> 172.21.15.46:0/229773527 conn(0x55ee3a8ce700 msgr2=0x55ee3a8ee390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:40.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.153+0000 7f87f5600640 0 client.26231 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:01:40.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.154+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 shutdown_connections 2024-04-17T08:01:40.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.154+0000 7f88159e8740 1 -- 172.21.15.46:0/229773527 wait complete. 2024-04-17T08:01:40.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.154+0000 7f88159e8740 20 client.26231 trim_cache size 0 max 16384 2024-04-17T08:01:40.165 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:01:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T08:01:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-17T08:01:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "97f91266-620b-4a89-aa0a-9798c84dbe40": { 2024-04-17T08:01:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-17T08:01:40.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "97f91266-620b-4a89-aa0a-9798c84dbe40", 2024-04-17T08:01:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-04-17T08:01:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:01:40.166 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:01:40.166 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f87f00a3cb0 msgr2=0x7f87f00a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f87f00a3cb0 0x7f87f00a6170 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006010 tx=0x7f87f802df90 comp rx=0 tx=0).stop 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 msgr2=0x7f880c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 0x7f880c19a4b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e9c0 tx=0x7f87fc00ee80 comp rx=0 tx=0).stop 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 shutdown_connections 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f87f00a3cb0 0x7f87f00a6170 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f880c105d60 0x7f880c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f880c1053a0 0x7f880c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 --2- 172.21.15.46:0/1447660801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f880c1041b0 0x7f880c19a4b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 >> 172.21.15.46:0/1447660801 conn(0x7f880c0ff720 msgr2=0x7f880c108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 shutdown_connections 2024-04-17T08:01:40.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:01:40.156+0000 7f8812c00640 1 -- 172.21.15.46:0/1447660801 wait complete. 2024-04-17T08:01:40.180 INFO:tasks.cephfs.filesystem:scrub status for tag:97f91266-620b-4a89-aa0a-9798c84dbe40 - {'path': '/', 'tag': '97f91266-620b-4a89-aa0a-9798c84dbe40', 'options': 'recursive,force'} 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: pgmap v512: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 165 KiB/s wr, 20 op/s 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: scrub queued for path: / 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: scrub summary: active paths [/] 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:40 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: pgmap v512: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 165 KiB/s wr, 20 op/s 2024-04-17T08:01:41.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: scrub queued for path: / 2024-04-17T08:01:41.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:41.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:41.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: scrub summary: active paths [/] 2024-04-17T08:01:41.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:40 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: pgmap v512: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 165 KiB/s wr, 20 op/s 2024-04-17T08:01:41.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: scrub queued for path: / 2024-04-17T08:01:41.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:41.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:41.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: scrub summary: active paths [/] 2024-04-17T08:01:41.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.360 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:40 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:41.484 INFO:tasks.workunit.client.0.smithi046.stdout: [DEP] 2024-04-17T08:01:41.512 INFO:tasks.workunit.client.0.smithi046.stdout: [DEP] 2024-04-17T08:01:41.513 INFO:tasks.workunit.client.0.smithi046.stdout: [DEP] 2024-04-17T08:01:42.028 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] dirstress 2024-04-17T08:01:42.034 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] fill 2024-04-17T08:01:42.035 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] fill2 2024-04-17T08:01:42.036 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] getpagesize 2024-04-17T08:01:42.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:41 smithi053 ceph-mon[36437]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:42.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:41 smithi062 ceph-mon[36507]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:42.353 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] holes 2024-04-17T08:01:42.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:41 smithi046 ceph-mon[32038]: scrub summary: idle+waiting paths [/] 2024-04-17T08:01:42.370 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] lstat64 2024-04-17T08:01:42.489 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] nametest 2024-04-17T08:01:42.490 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] permname 2024-04-17T08:01:42.748 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] randholes 2024-04-17T08:01:42.769 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] runas 2024-04-17T08:01:42.881 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] truncfile 2024-04-17T08:01:42.957 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] usemem 2024-04-17T08:01:43.169 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] mmapcat 2024-04-17T08:01:43.201 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] append_reader 2024-04-17T08:01:43.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:42 smithi053 ceph-mon[36437]: pgmap v513: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-04-17T08:01:43.246 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] append_writer 2024-04-17T08:01:43.259 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] dirperf 2024-04-17T08:01:43.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:42 smithi062 ceph-mon[36507]: pgmap v513: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-04-17T08:01:43.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:42 smithi046 ceph-mon[32038]: pgmap v513: 129 pgs: 129 active+clean; 69 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-04-17T08:01:43.474 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] metaperf 2024-04-17T08:01:43.508 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] devzero 2024-04-17T08:01:43.542 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] feature 2024-04-17T08:01:43.680 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] alloc 2024-04-17T08:01:43.904 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] fault 2024-04-17T08:01:43.943 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] fstest 2024-04-17T08:01:43.945 INFO:tasks.workunit.client.0.smithi046.stdout: [CC] t_access_root 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr:alloc.c: In function 'main': 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr: 154 | bozo! 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~ 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr: 154 | bozo! 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr: | ; 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-04-17T08:01:43.951 INFO:tasks.workunit.client.0.smithi046.stderr: 184 | rflag = 1; 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~ 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr: | nflag 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr: 320 | bozo! 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr: | ; 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-04-17T08:01:43.952 INFO:tasks.workunit.client.0.smithi046.stderr: 128 | struct flock64 f; 2024-04-17T08:01:43.953 INFO:tasks.workunit.client.0.smithi046.stderr: | ^ 2024-04-17T08:01:43.960 INFO:tasks.workunit.client.0.smithi046.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-04-17T08:01:43.960 INFO:tasks.workunit.client.0.smithi046.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-17T08:01:44.222 INFO:tasks.workunit.client.0.smithi046.stderr:fstest.c: In function 'run_child': 2024-04-17T08:01:44.222 INFO:tasks.workunit.client.0.smithi046.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~ 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~ 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~ 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-17T08:01:44.223 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~ 2024-04-17T08:01:44.224 INFO:tasks.workunit.client.0.smithi046.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-17T08:01:44.224 INFO:tasks.workunit.client.0.smithi046.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-17T08:01:44.224 INFO:tasks.workunit.client.0.smithi046.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-17T08:01:44.335 INFO:tasks.workunit.client.0.smithi046.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-04-17T08:01:44.335 INFO:tasks.workunit.client.0.smithi046.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-04-17T08:01:44.336 INFO:tasks.workunit.client.0.smithi046.stderr:make: *** [Makefile:51: default] Error 2 2024-04-17T08:01:44.337 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-17T08:01:44.338 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-17T08:01:44.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-17T08:01:45.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:44 smithi053 ceph-mon[36437]: pgmap v514: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 24 op/s 2024-04-17T08:01:45.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:44 smithi062 ceph-mon[36507]: pgmap v514: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 24 op/s 2024-04-17T08:01:45.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:44 smithi046 ceph-mon[32038]: pgmap v514: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 24 op/s 2024-04-17T08:01:45.398 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi046 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-17T08:01:45.643 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=09e02bd6081744cdb6e54d9be0db1ba4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi046 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-17T08:01:45.646 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-17T08:01:45.655 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-17T08:01:47.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:46 smithi053 ceph-mon[36437]: pgmap v515: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 23 op/s 2024-04-17T08:01:47.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:46 smithi062 ceph-mon[36507]: pgmap v515: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 23 op/s 2024-04-17T08:01:47.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:46 smithi046 ceph-mon[32038]: pgmap v515: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 23 op/s 2024-04-17T08:01:49.222 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:48 smithi053 ceph-mon[36437]: pgmap v516: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-17T08:01:49.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:48 smithi062 ceph-mon[36507]: pgmap v516: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-17T08:01:49.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:48 smithi046 ceph-mon[32038]: pgmap v516: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-17T08:01:51.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:50 smithi053 ceph-mon[36437]: pgmap v517: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-17T08:01:51.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:50 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:01:51.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:50 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:01:51.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:50 smithi062 ceph-mon[36507]: pgmap v517: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-17T08:01:51.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:50 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:01:51.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:50 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:01:51.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:50 smithi046 ceph-mon[32038]: pgmap v517: 129 pgs: 129 active+clean; 70 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-04-17T08:01:51.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:50 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:01:51.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:50 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:01:52.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:51 smithi053 ceph-mon[36437]: scrub summary: idle 2024-04-17T08:01:52.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:51 smithi062 ceph-mon[36507]: scrub summary: idle 2024-04-17T08:01:52.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:51 smithi046 ceph-mon[32038]: scrub summary: idle 2024-04-17T08:01:53.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:52 smithi053 ceph-mon[36437]: pgmap v518: 129 pgs: 129 active+clean; 71 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 206 KiB/s wr, 49 op/s 2024-04-17T08:01:53.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:52 smithi062 ceph-mon[36507]: pgmap v518: 129 pgs: 129 active+clean; 71 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 206 KiB/s wr, 49 op/s 2024-04-17T08:01:53.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:52 smithi046 ceph-mon[32038]: pgmap v518: 129 pgs: 129 active+clean; 71 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 206 KiB/s wr, 49 op/s 2024-04-17T08:01:55.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:54 smithi053 ceph-mon[36437]: pgmap v519: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 341 KiB/s wr, 60 op/s 2024-04-17T08:01:55.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:54 smithi062 ceph-mon[36507]: pgmap v519: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 341 KiB/s wr, 60 op/s 2024-04-17T08:01:55.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:54 smithi046 ceph-mon[32038]: pgmap v519: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 341 KiB/s wr, 60 op/s 2024-04-17T08:01:57.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:56 smithi053 ceph-mon[36437]: pgmap v520: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 328 KiB/s wr, 53 op/s 2024-04-17T08:01:57.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:56 smithi062 ceph-mon[36507]: pgmap v520: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 328 KiB/s wr, 53 op/s 2024-04-17T08:01:57.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:56 smithi046 ceph-mon[32038]: pgmap v520: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 328 KiB/s wr, 53 op/s 2024-04-17T08:01:59.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:01:58 smithi053 ceph-mon[36437]: pgmap v521: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 330 KiB/s wr, 55 op/s 2024-04-17T08:01:59.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:01:58 smithi062 ceph-mon[36507]: pgmap v521: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 330 KiB/s wr, 55 op/s 2024-04-17T08:01:59.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:01:58 smithi046 ceph-mon[32038]: pgmap v521: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 330 KiB/s wr, 55 op/s 2024-04-17T08:02:01.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:00 smithi062 ceph-mon[36507]: pgmap v522: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 326 KiB/s wr, 50 op/s 2024-04-17T08:02:01.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:00 smithi046 ceph-mon[32038]: pgmap v522: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 326 KiB/s wr, 50 op/s 2024-04-17T08:02:01.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:00 smithi053 ceph-mon[36437]: pgmap v522: 129 pgs: 129 active+clean; 72 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 326 KiB/s wr, 50 op/s 2024-04-17T08:02:02.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:01 smithi062 ceph-mon[36507]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:02:02.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:01 smithi046 ceph-mon[32038]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:02:02.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:01 smithi053 ceph-mon[36437]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:02:03.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:02 smithi062 ceph-mon[36507]: pgmap v523: 129 pgs: 129 active+clean; 73 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 335 KiB/s wr, 50 op/s 2024-04-17T08:02:03.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:02 smithi046 ceph-mon[32038]: pgmap v523: 129 pgs: 129 active+clean; 73 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 335 KiB/s wr, 50 op/s 2024-04-17T08:02:03.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:02 smithi053 ceph-mon[36437]: pgmap v523: 129 pgs: 129 active+clean; 73 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 335 KiB/s wr, 50 op/s 2024-04-17T08:02:05.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:05 smithi062 ceph-mon[36507]: pgmap v524: 129 pgs: 129 active+clean; 73 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 183 KiB/s wr, 29 op/s 2024-04-17T08:02:05.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:05 smithi046 ceph-mon[32038]: pgmap v524: 129 pgs: 129 active+clean; 73 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 183 KiB/s wr, 29 op/s 2024-04-17T08:02:05.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:05 smithi053 ceph-mon[36437]: pgmap v524: 129 pgs: 129 active+clean; 73 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 183 KiB/s wr, 29 op/s 2024-04-17T08:02:06.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:06 smithi062 ceph-mon[36507]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:02:06.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:06 smithi046 ceph-mon[32038]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:02:06.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:06 smithi053 ceph-mon[36437]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-17T08:02:07.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:07 smithi062 ceph-mon[36507]: pgmap v525: 129 pgs: 129 active+clean; 73 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-17T08:02:07.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:07 smithi046 ceph-mon[32038]: pgmap v525: 129 pgs: 129 active+clean; 73 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-17T08:02:07.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:07 smithi053 ceph-mon[36437]: pgmap v525: 129 pgs: 129 active+clean; 73 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-04-17T08:02:09.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:09 smithi062 ceph-mon[36507]: pgmap v526: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-04-17T08:02:09.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:09 smithi046 ceph-mon[32038]: pgmap v526: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-04-17T08:02:09.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:09 smithi053 ceph-mon[36437]: pgmap v526: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-04-17T08:02:10.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T08:02:10.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T08:02:10.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 -- 172.21.15.46:0/2230112268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74102850 msgr2=0x7f6c74104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/2230112268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74102850 0x7f6c74104c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009860 tx=0x7f6c6802bb70 comp rx=0 tx=0).stop 2024-04-17T08:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 -- 172.21.15.46:0/2230112268 shutdown_connections 2024-04-17T08:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/2230112268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c74107ab0 0x7f6c74109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/2230112268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74105180 0x7f6c74107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/2230112268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74102850 0x7f6c74104c40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 -- 172.21.15.46:0/2230112268 >> 172.21.15.46:0/2230112268 conn(0x7f6c740fc240 msgr2=0x7f6c740fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 -- 172.21.15.46:0/2230112268 shutdown_connections 2024-04-17T08:02:10.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.423+0000 7f6c7aa00640 1 -- 172.21.15.46:0/2230112268 wait complete. 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 Processor -- start 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 -- start start 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74102850 0x7f6c7419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74105180 0x7f6c7419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c74107ab0 0x7f6c741a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410da50 con 0x7f6c74102850 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410d8d0 con 0x7f6c74107ab0 2024-04-17T08:02:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c7aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410dbd0 con 0x7f6c74105180 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.425+0000 7f6c79600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74102850 0x7f6c7419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74102850 0x7f6c7419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43994/0 (socket says 172.21.15.46:43994) 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 -- 172.21.15.46:0/1747353288 learned_addr learned my addr 172.21.15.46:0/1747353288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c74107ab0 0x7f6c741a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c78c00640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74105180 0x7f6c7419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 -- 172.21.15.46:0/1747353288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74105180 msgr2=0x7f6c7419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74105180 0x7f6c7419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 -- 172.21.15.46:0/1747353288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c74107ab0 msgr2=0x7f6c741a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c74107ab0 0x7f6c741a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 -- 172.21.15.46:0/1747353288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741a1f20 con 0x7f6c74102850 2024-04-17T08:02:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.426+0000 7f6c79600640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74102850 0x7f6c7419a300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6c680349e0 tx=0x7f6c68032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.427+0000 7f6c72000640 1 -- 172.21.15.46:0/1747353288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68041030 con 0x7f6c74102850 2024-04-17T08:02:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.427+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741a21b0 con 0x7f6c74102850 2024-04-17T08:02:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.427+0000 7f6c72000640 1 -- 172.21.15.46:0/1747353288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6802c510 con 0x7f6c74102850 2024-04-17T08:02:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.427+0000 7f6c72000640 1 -- 172.21.15.46:0/1747353288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68039040 con 0x7f6c74102850 2024-04-17T08:02:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.427+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a26a0 con 0x7f6c74102850 2024-04-17T08:02:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.431+0000 7f6c7ed99740 1 Processor -- start 2024-04-17T08:02:10.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.431+0000 7f6c7ed99740 1 -- start start 2024-04-17T08:02:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a1a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1fed0 0x559e49a22360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e49981120 con 0x559e49a1a370 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559e49927340 con 0x559e49a1ad60 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559e4997d0e0 con 0x559e49a1fed0 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c72000640 1 -- 172.21.15.46:0/1747353288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6c68044070 con 0x7f6c74102850 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a1a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a1a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44004/0 (socket says 172.21.15.46:44004) 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c7a000640 1 -- 172.21.15.46:0/2586882870 learned_addr learned my addr 172.21.15.46:0/2586882870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c78c00640 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1fed0 0x559e49a22360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.432+0000 7f6c79600640 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c7a000640 1 -- 172.21.15.46:0/2586882870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1fed0 msgr2=0x559e49a22360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c7a000640 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1fed0 0x559e49a22360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c7a000640 1 -- 172.21.15.46:0/2586882870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 msgr2=0x559e49a1f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c7a000640 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a1f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c7a000640 1 -- 172.21.15.46:0/2586882870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e4975d7d0 con 0x559e49a1a370 2024-04-17T08:02:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c7a000640 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a1a790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e580 tx=0x7f6c6c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66470c75ec21c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c66600640 1 -- 172.21.15.46:0/2586882870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c003970 con 0x559e49a1a370 2024-04-17T08:02:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c66600640 1 -- 172.21.15.46:0/2586882870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c00d830 con 0x559e49a1a370 2024-04-17T08:02:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.433+0000 7f6c66600640 1 -- 172.21.15.46:0/2586882870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004560 con 0x559e49a1a370 2024-04-17T08:02:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c72000640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c5c0a3bb0 0x7f6c5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c72000640 1 -- 172.21.15.46:0/1747353288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f6c680f1d10 con 0x7f6c74102850 2024-04-17T08:02:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c79600640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c5c0a3bb0 0x7f6c5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2586882870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 msgr2=0x559e49a1a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a1a790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e580 tx=0x7f6c6c0095c0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2586882870 shutdown_connections 2024-04-17T08:02:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1fed0 0x559e49a22360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a1f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2586882870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a1a790 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2586882870 >> 172.21.15.46:0/2586882870 conn(0x559e49a158e0 msgr2=0x559e49a15ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.434+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2586882870 shutdown_connections 2024-04-17T08:02:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.435+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2586882870 wait complete. 2024-04-17T08:02:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.435+0000 7f6c79600640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c5c0a3bb0 0x7f6c5c0a6070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6c680bdb90 tx=0x7f6c680de040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.435+0000 7f6c7ed99740 1 -- start start 2024-04-17T08:02:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.435+0000 7f6c7ed99740 1 Processor -- start 2024-04-17T08:02:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a24850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1ad60 0x559e49a24d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a2bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e49a22e80 con 0x559e49a1ad60 2024-04-17T08:02:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559e4993a710 con 0x559e49a1fed0 2024-04-17T08:02:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559e49a23000 con 0x559e49a1a370 2024-04-17T08:02:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7ed99740 10 client.0 _ll_register_callbacks cb 0x559e49935af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c78c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1ad60 0x559e49a24d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7a000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a2bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c78c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1ad60 0x559e49a24d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44018/0 (socket says 172.21.15.46:44018) 2024-04-17T08:02:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c78c00640 1 -- 172.21.15.46:0/2052890465 learned_addr learned my addr 172.21.15.46:0/2052890465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c7a000640 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a24850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.436+0000 7f6c78c00640 1 -- 172.21.15.46:0/2052890465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 msgr2=0x559e49a24850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c78c00640 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a24850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c78c00640 1 -- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 msgr2=0x559e49a2bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c78c00640 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a2bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c78c00640 1 -- 172.21.15.46:0/2052890465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e496933f0 con 0x559e49a1ad60 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c7a000640 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a2bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c78c00640 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1ad60 0x559e49a24d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c60012700 tx=0x7f6c60012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502776bbdd2bdf29 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:02:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c5aa00640 1 -- 172.21.15.46:0/2052890465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000f8c0 con 0x559e49a1ad60 2024-04-17T08:02:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c5aa00640 1 -- 172.21.15.46:0/2052890465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f6c6000fa60 con 0x559e49a1ad60 2024-04-17T08:02:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c00d830 con 0x559e49a1ad60 2024-04-17T08:02:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.437+0000 7f6c5aa00640 1 -- 172.21.15.46:0/2052890465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c6000fc60 con 0x559e49a1ad60 2024-04-17T08:02:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 10 client.16071 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e496530f0 con 0x559e49a1ad60 2024-04-17T08:02:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c5aa00640 1 -- 172.21.15.46:0/2052890465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c00d830 con 0x559e49a1ad60 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c5aa00640 1 -- 172.21.15.46:0/2052890465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6c6001c4b0 con 0x559e49a1ad60 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 10 client.16071 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 20 client.16071 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a30860 0x559e49a32ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 4 client.16071 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.438+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e4998eb40 con 0x559e49a30860 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.439+0000 7f6c7a000640 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a30860 0x559e49a32ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.439+0000 7f6c7a000640 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a30860 0x559e49a32ca0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1b7d0d86f0bdeb server_cookie=1326c460dfb2bc56 in_seq=0 out_seq=0 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.439+0000 7f6c5aa00640 10 client.16071 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c5aa00640 1 -- 172.21.15.46:0/2052890465 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e4998eb40 con 0x559e49a30860 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c5aa00640 10 client.16071 handle_command_reply: tid=0 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 client.16071 shutdown 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1ad60 msgr2=0x559e49a24d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1ad60 0x559e49a24d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c60012700 tx=0x7f6c60012bc0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 shutdown_connections 2024-04-17T08:02:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a30860 0x559e49a32ca0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a2bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1ad60 0x559e49a24d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/2052890465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a24850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c5aa00640 0 client.16071 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 >> 172.21.15.46:0/2052890465 conn(0x559e49a158e0 msgr2=0x559e49a14ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.440+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 shutdown_connections 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.441+0000 7f6c7ed99740 1 -- 172.21.15.46:0/2052890465 wait complete. 2024-04-17T08:02:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.441+0000 7f6c7ed99740 20 client.16071 trim_cache size 0 max 16384 2024-04-17T08:02:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 Processor -- start 2024-04-17T08:02:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 -- start start 2024-04-17T08:02:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a10970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1ad60 0x559e49a2bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a14250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741af560 con 0x559e49a1a370 2024-04-17T08:02:10.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741acdc0 con 0x559e49a1fed0 2024-04-17T08:02:10.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.460+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741aac10 con 0x559e49a1ad60 2024-04-17T08:02:10.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a10970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a10970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44026/0 (socket says 172.21.15.46:44026) 2024-04-17T08:02:10.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 -- 172.21.15.46:0/1939943854 learned_addr learned my addr 172.21.15.46:0/1939943854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:10.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c78c00640 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1ad60 0x559e49a2bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a14250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 -- 172.21.15.46:0/1939943854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1ad60 msgr2=0x559e49a2bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1ad60 0x559e49a2bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 -- 172.21.15.46:0/1939943854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 msgr2=0x559e49a14250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a14250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 -- 172.21.15.46:0/1939943854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e49a16350 con 0x559e49a1a370 2024-04-17T08:02:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c78c00640 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1ad60 0x559e49a2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.461+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a10970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e520 tx=0x7f6c6c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bf5ea042c0f5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c65c00640 1 -- 172.21.15.46:0/1939943854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c003a70 con 0x559e49a1a370 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c65c00640 1 -- 172.21.15.46:0/1939943854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c02d720 con 0x559e49a1a370 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c65c00640 1 -- 172.21.15.46:0/1939943854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0053a0 con 0x559e49a1a370 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1939943854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 msgr2=0x559e49a10970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a10970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e520 tx=0x7f6c6c005fd0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1939943854 shutdown_connections 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1fed0 0x559e49a14250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1ad60 0x559e49a2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1939943854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1a370 0x559e49a10970 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.462+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1939943854 >> 172.21.15.46:0/1939943854 conn(0x559e49a158e0 msgr2=0x559e49a1c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.463+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1939943854 shutdown_connections 2024-04-17T08:02:10.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.463+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1939943854 wait complete. 2024-04-17T08:02:10.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.463+0000 7f6c7ed99740 1 -- start start 2024-04-17T08:02:10.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.463+0000 7f6c7ed99740 1 Processor -- start 2024-04-17T08:02:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7ed99740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1fed0 0x559e49a2aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741ab280 con 0x559e49a1fed0 2024-04-17T08:02:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741ac6f0 con 0x559e49a1ad60 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7ed99740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741ab950 con 0x559e49a1a370 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7ed99740 10 client.0 _ll_register_callbacks cb 0x7f6c60009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1fed0 0x559e49a2aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a29a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47242/0 (socket says 172.21.15.46:47242) 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 -- 172.21.15.46:0/1186929769 learned_addr learned my addr 172.21.15.46:0/1186929769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 -- 172.21.15.46:0/1186929769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 msgr2=0x559e49a294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 -- 172.21.15.46:0/1186929769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1fed0 msgr2=0x559e49a2aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1fed0 0x559e49a2aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c78c00640 1 -- 172.21.15.46:0/1186929769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e49a2b010 con 0x559e49a1ad60 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.464+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c78c00640 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a29a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x559e49a25d30 tx=0x7f6c60009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9edc1fe69032191 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c5aa00640 1 -- 172.21.15.46:0/1186929769 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6002e030 con 0x559e49a1ad60 2024-04-17T08:02:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c5aa00640 1 -- 172.21.15.46:0/1186929769 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f6c60018050 con 0x559e49a1ad60 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c02d720 con 0x559e49a1ad60 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c5aa00640 1 -- 172.21.15.46:0/1186929769 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c6001c420 con 0x559e49a1ad60 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c7ed99740 10 client.26249 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.465+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e49a96ee0 con 0x559e49a1ad60 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c5aa00640 1 -- 172.21.15.46:0/1186929769 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c02d720 con 0x559e49a1ad60 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c5aa00640 1 -- 172.21.15.46:0/1186929769 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6c60003710 con 0x559e49a1ad60 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c7ed99740 10 client.26249 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c7ed99740 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:02:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c7ed99740 20 client.26249 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a97910 0x559e49a99d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c7ed99740 4 client.26249 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T08:02:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.466+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e49a11140 con 0x559e49a97910 2024-04-17T08:02:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a97910 0x559e49a99d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c7a000640 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a97910 0x559e49a99d30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d9ef3d8a6c4c74 server_cookie=47f90735bcadc663 in_seq=0 out_seq=0 2024-04-17T08:02:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c5aa00640 10 client.26249 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c5aa00640 1 -- 172.21.15.46:0/1186929769 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e49a11140 con 0x559e49a97910 2024-04-17T08:02:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c5aa00640 10 client.26249 handle_command_reply: tid=0 2024-04-17T08:02:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c7ed99740 1 client.26249 shutdown 2024-04-17T08:02:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 msgr2=0x559e49a29a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.467+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a29a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x559e49a25d30 tx=0x7f6c60009ee0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 shutdown_connections 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x559e49a97910 0x559e49a99d30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e49a1fed0 0x559e49a2aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c5aa00640 0 client.26249 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559e49a1ad60 0x559e49a29a10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 --2- 172.21.15.46:0/1186929769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e49a1a370 0x559e49a294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 >> 172.21.15.46:0/1186929769 conn(0x559e49a158e0 msgr2=0x559e49a18b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 shutdown_connections 2024-04-17T08:02:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 1 -- 172.21.15.46:0/1186929769 wait complete. 2024-04-17T08:02:10.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.468+0000 7f6c7ed99740 20 client.26249 trim_cache size 0 max 16384 2024-04-17T08:02:10.477 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:02:10.477 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-17T08:02:10.477 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-17T08:02:10.477 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:02:10.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c5c0a3bb0 msgr2=0x7f6c5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c5c0a3bb0 0x7f6c5c0a6070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6c680bdb90 tx=0x7f6c680de040 comp rx=0 tx=0).stop 2024-04-17T08:02:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74102850 msgr2=0x7f6c7419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74102850 0x7f6c7419a300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6c680349e0 tx=0x7f6c68032450 comp rx=0 tx=0).stop 2024-04-17T08:02:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 shutdown_connections 2024-04-17T08:02:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6c5c0a3bb0 0x7f6c5c0a6070 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c74107ab0 0x7f6c741a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c74105180 0x7f6c7419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 --2- 172.21.15.46:0/1747353288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c74102850 0x7f6c7419a300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 >> 172.21.15.46:0/1747353288 conn(0x7f6c740fc240 msgr2=0x7f6c74101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 shutdown_connections 2024-04-17T08:02:10.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.469+0000 7f6c7aa00640 1 -- 172.21.15.46:0/1747353288 wait complete. 2024-04-17T08:02:10.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T08:02:10.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T08:02:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.714+0000 7fa857e00640 1 -- 172.21.15.46:0/3344665543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858105a30 msgr2=0x7fa858105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.714+0000 7fa857e00640 1 --2- 172.21.15.46:0/3344665543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858105a30 0x7fa858105eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa850009ca0 tx=0x7fa85002c100 comp rx=0 tx=0).stop 2024-04-17T08:02:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.714+0000 7fa857e00640 1 -- 172.21.15.46:0/3344665543 shutdown_connections 2024-04-17T08:02:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.714+0000 7fa857e00640 1 --2- 172.21.15.46:0/3344665543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858105a30 0x7fa858105eb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.714+0000 7fa857e00640 1 --2- 172.21.15.46:0/3344665543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858103080 0x7fa8581054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.714+0000 7fa857e00640 1 --2- 172.21.15.46:0/3344665543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858100710 0x7fa858102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.714+0000 7fa857e00640 1 -- 172.21.15.46:0/3344665543 >> 172.21.15.46:0/3344665543 conn(0x7fa8580fc330 msgr2=0x7fa8580fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.715+0000 7fa857e00640 1 -- 172.21.15.46:0/3344665543 shutdown_connections 2024-04-17T08:02:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.715+0000 7fa857e00640 1 -- 172.21.15.46:0/3344665543 wait complete. 2024-04-17T08:02:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 Processor -- start 2024-04-17T08:02:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 -- start start 2024-04-17T08:02:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858100710 0x7fa85819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858103080 0x7fa85819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858105a30 0x7fa8581a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa858114140 con 0x7fa858105a30 2024-04-17T08:02:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa858113fc0 con 0x7fa858103080 2024-04-17T08:02:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.716+0000 7fa857e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581142c0 con 0x7fa858100710 2024-04-17T08:02:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa857400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858105a30 0x7fa8581a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858100710 0x7fa85819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa857400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858105a30 0x7fa8581a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44058/0 (socket says 172.21.15.46:44058) 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa857400640 1 -- 172.21.15.46:0/2546651248 learned_addr learned my addr 172.21.15.46:0/2546651248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856000640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858103080 0x7fa85819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856000640 1 -- 172.21.15.46:0/2546651248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858100710 msgr2=0x7fa85819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856000640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858100710 0x7fa85819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856000640 1 -- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858105a30 msgr2=0x7fa8581a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856000640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858105a30 0x7fa8581a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856000640 1 -- 172.21.15.46:0/2546651248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581a6340 con 0x7fa858103080 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa856a00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858100710 0x7fa85819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.717+0000 7fa857400640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858105a30 0x7fa8581a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.718+0000 7fa856000640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858103080 0x7fa85819ec60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e9f0 tx=0x7fa84400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.718+0000 7fa85ca00640 1 -- 172.21.15.46:0/2546651248 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84400ccc0 con 0x7fa858103080 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.718+0000 7fa85ca00640 1 -- 172.21.15.46:0/2546651248 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844010910 con 0x7fa858103080 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.718+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581a6630 con 0x7fa858103080 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.718+0000 7fa85ca00640 1 -- 172.21.15.46:0/2546651248 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844018960 con 0x7fa858103080 2024-04-17T08:02:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.718+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581a6b80 con 0x7fa858103080 2024-04-17T08:02:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.719+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8580fde20 con 0x7fa858103080 2024-04-17T08:02:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.722+0000 7fa85ca00640 1 -- 172.21.15.46:0/2546651248 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa844010ac0 con 0x7fa858103080 2024-04-17T08:02:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.724+0000 7fa85ca00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3ba0 0x7fa83c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.724+0000 7fa85ca00640 1 -- 172.21.15.46:0/2546651248 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa8440d0fe0 con 0x7fa858103080 2024-04-17T08:02:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.725+0000 7fa856a00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3ba0 0x7fa83c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.725+0000 7fa856a00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3ba0 0x7fa83c0a6060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa848009fc0 tx=0x7fa848005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.725+0000 7fa85ca00640 1 -- 172.21.15.46:0/2546651248 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa84409cfc0 con 0x7fa858103080 2024-04-17T08:02:10.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.978+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa85802d820 con 0x7fa858103080 2024-04-17T08:02:10.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.979+0000 7fa85ca00640 1 -- 172.21.15.46:0/2546651248 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11875 (secure 0 0 0) 0x7fa8440a12c0 con 0x7fa858103080 2024-04-17T08:02:10.981 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:02:10.981 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T08:02:04.995472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T08:02:10.983 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-17T08:02:10.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3ba0 msgr2=0x7fa83c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3ba0 0x7fa83c0a6060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa848009fc0 tx=0x7fa848005f90 comp rx=0 tx=0).stop 2024-04-17T08:02:10.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858103080 msgr2=0x7fa85819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:10.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858103080 0x7fa85819ec60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e9f0 tx=0x7fa84400eeb0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 shutdown_connections 2024-04-17T08:02:10.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa83c0a3ba0 0x7fa83c0a6060 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa858105a30 0x7fa8581a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858103080 0x7fa85819ec60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 --2- 172.21.15.46:0/2546651248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa858100710 0x7fa85819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:10.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.982+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 >> 172.21.15.46:0/2546651248 conn(0x7fa8580fc330 msgr2=0x7fa8581013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:10.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.983+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 shutdown_connections 2024-04-17T08:02:10.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:10.983+0000 7fa857e00640 1 -- 172.21.15.46:0/2546651248 wait complete. 2024-04-17T08:02:11.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T08:02:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.230+0000 7f2692000640 1 -- 172.21.15.46:0/3321726763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c102fe0 msgr2=0x7f268c103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.230+0000 7f2692000640 1 --2- 172.21.15.46:0/3321726763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c102fe0 0x7f268c103460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2674009900 tx=0x7f267402bc50 comp rx=0 tx=0).stop 2024-04-17T08:02:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.231+0000 7f2692000640 1 -- 172.21.15.46:0/3321726763 shutdown_connections 2024-04-17T08:02:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.231+0000 7f2692000640 1 --2- 172.21.15.46:0/3321726763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c1039a0 0x7f268c10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.231+0000 7f2692000640 1 --2- 172.21.15.46:0/3321726763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c102fe0 0x7f268c103460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.231+0000 7f2692000640 1 --2- 172.21.15.46:0/3321726763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c101df0 0x7f268c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.231+0000 7f2692000640 1 -- 172.21.15.46:0/3321726763 >> 172.21.15.46:0/3321726763 conn(0x7f268c0fd360 msgr2=0x7f268c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.231+0000 7f2692000640 1 -- 172.21.15.46:0/3321726763 shutdown_connections 2024-04-17T08:02:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.231+0000 7f2692000640 1 -- 172.21.15.46:0/3321726763 wait complete. 2024-04-17T08:02:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 Processor -- start 2024-04-17T08:02:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 -- start start 2024-04-17T08:02:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c101df0 0x7f268c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c102fe0 0x7f268c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c1039a0 0x7f268c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f268c10daa0 con 0x7f268c101df0 2024-04-17T08:02:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f268c10d920 con 0x7f268c1039a0 2024-04-17T08:02:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2692000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f268c10dc20 con 0x7f268c102fe0 2024-04-17T08:02:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.232+0000 7f2690c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c101df0 0x7f268c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c102fe0 0x7f268c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2691600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c1039a0 0x7f268c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c102fe0 0x7f268c19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:43386/0 (socket says 172.21.15.46:43386) 2024-04-17T08:02:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 -- 172.21.15.46:0/3578566635 learned_addr learned my addr 172.21.15.46:0/3578566635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 -- 172.21.15.46:0/3578566635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c1039a0 msgr2=0x7f268c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c1039a0 0x7f268c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 -- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c101df0 msgr2=0x7f268c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c101df0 0x7f268c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 -- 172.21.15.46:0/3578566635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c1a1ed0 con 0x7f268c102fe0 2024-04-17T08:02:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2690c00640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c101df0 0x7f268c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.233+0000 7f2683e00640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c102fe0 0x7f268c19a7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f267402c260 tx=0x7f267402c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.234+0000 7f268a000640 1 -- 172.21.15.46:0/3578566635 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674041070 con 0x7f268c102fe0 2024-04-17T08:02:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.234+0000 7f268a000640 1 -- 172.21.15.46:0/3578566635 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2674004450 con 0x7f268c102fe0 2024-04-17T08:02:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.234+0000 7f268a000640 1 -- 172.21.15.46:0/3578566635 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674034440 con 0x7f268c102fe0 2024-04-17T08:02:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.234+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c1a2160 con 0x7f268c102fe0 2024-04-17T08:02:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.234+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1a2660 con 0x7f268c102fe0 2024-04-17T08:02:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.237+0000 7f268a000640 1 -- 172.21.15.46:0/3578566635 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2674044070 con 0x7f268c102fe0 2024-04-17T08:02:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.237+0000 7f269640e740 1 Processor -- start 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.237+0000 7f269640e740 1 -- start start 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f269640e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d15160 0x5649e6d15580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f269640e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d1a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f269640e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d1acc0 0x5649e6d1d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f269640e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5649e6c7b020 con 0x5649e6d1acc0 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f269640e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5649e6c21460 con 0x5649e6d15160 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f269640e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5649e6c76fe0 con 0x5649e6d15b50 2024-04-17T08:02:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2691600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d15160 0x5649e6d15580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2683e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d1acc0 0x5649e6d1d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2691600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d15160 0x5649e6d15580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47296/0 (socket says 172.21.15.46:47296) 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2691600640 1 -- 172.21.15.46:0/1488791864 learned_addr learned my addr 172.21.15.46:0/1488791864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2690c00640 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d1a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2691600640 1 -- 172.21.15.46:0/1488791864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 msgr2=0x5649e6d1a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2691600640 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d1a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2691600640 1 -- 172.21.15.46:0/1488791864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d1acc0 msgr2=0x5649e6d1d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.238+0000 7f2691600640 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d1acc0 0x5649e6d1d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f2691600640 1 -- 172.21.15.46:0/1488791864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e6c29040 con 0x5649e6d15160 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f2690c00640 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d1a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f268a000640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26700a3bb0 0x7f26700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f2691600640 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d15160 0x5649e6d15580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2684009ca0 tx=0x7f2684011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66692bc7ebcfb11b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f268a000640 1 -- 172.21.15.46:0/3578566635 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f26740f2280 con 0x7f268c102fe0 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f2690c00640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26700a3bb0 0x7f26700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f2681c00640 1 -- 172.21.15.46:0/1488791864 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268401e070 con 0x5649e6d15160 2024-04-17T08:02:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f2681c00640 1 -- 172.21.15.46:0/1488791864 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2684009e50 con 0x5649e6d15160 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.239+0000 7f2681c00640 1 -- 172.21.15.46:0/1488791864 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268400ea80 con 0x5649e6d15160 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f2690c00640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26700a3bb0 0x7f26700a6070 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5649e6d15f50 tx=0x7f267c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 -- 172.21.15.46:0/1488791864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d15160 msgr2=0x5649e6d15580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d15160 0x5649e6d15580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2684009ca0 tx=0x7f2684011810 comp rx=0 tx=0).stop 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 -- 172.21.15.46:0/1488791864 shutdown_connections 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d1acc0 0x5649e6d1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d1a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 --2- 172.21.15.46:0/1488791864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d15160 0x5649e6d15580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 -- 172.21.15.46:0/1488791864 >> 172.21.15.46:0/1488791864 conn(0x5649e6d106d0 msgr2=0x5649e6d10ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 -- 172.21.15.46:0/1488791864 shutdown_connections 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.240+0000 7f269640e740 1 -- 172.21.15.46:0/1488791864 wait complete. 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.241+0000 7f269640e740 1 -- start start 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.241+0000 7f269640e740 1 Processor -- start 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.241+0000 7f269640e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d1f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.241+0000 7f269640e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d1fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.241+0000 7f269640e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d26cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.241+0000 7f269640e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5649e6d1dc70 con 0x5649e6d15b50 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f269640e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5649e6c34830 con 0x5649e6d1acc0 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f269640e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5649e6d1ddf0 con 0x5649e6d15160 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f269640e740 10 client.0 _ll_register_callbacks cb 0x5649e6c02fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d1fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d1fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44096/0 (socket says 172.21.15.46:44096) 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 -- 172.21.15.46:0/3366939646 learned_addr learned my addr 172.21.15.46:0/3366939646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2690c00640 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d1f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2691600640 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d26cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 -- 172.21.15.46:0/3366939646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 msgr2=0x5649e6d1f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d1f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 -- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 msgr2=0x5649e6d26cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d26cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.242+0000 7f2683e00640 1 -- 172.21.15.46:0/3366939646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e6c38920 con 0x5649e6d15b50 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f2683e00640 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d1fc70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f26740ba680 tx=0x7f26740f57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff63d497e8b18a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f267a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f267a000640 1 -- 172.21.15.46:0/3366939646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267404b070 con 0x5649e6d15b50 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f267a000640 1 -- 172.21.15.46:0/3366939646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f26740c6950 con 0x5649e6d15b50 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2684009e50 con 0x5649e6d15b50 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f267a000640 1 -- 172.21.15.46:0/3366939646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26740ba4c0 con 0x5649e6d15b50 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f269640e740 10 client.16095 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649e6c067f0 con 0x5649e6d15b50 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f267a000640 1 -- 172.21.15.46:0/3366939646 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2684009e50 con 0x5649e6d15b50 2024-04-17T08:02:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.243+0000 7f267a000640 1 -- 172.21.15.46:0/3366939646 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f26740c2170 con 0x5649e6d15b50 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.244+0000 7f269640e740 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.244+0000 7f269640e740 10 client.16095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.244+0000 7f269640e740 20 client.16095 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.244+0000 7f269640e740 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d2b190 0x5649e6d2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.244+0000 7f269640e740 4 client.16095 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.244+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649e6c88a40 con 0x5649e6d2b190 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.244+0000 7f2690c00640 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d2b190 0x5649e6d2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.245+0000 7f2690c00640 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d2b190 0x5649e6d2d610 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3242d17ab8bca6b server_cookie=2696c60f4eef9837 in_seq=0 out_seq=0 2024-04-17T08:02:11.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.245+0000 7f267a000640 10 client.16095 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:02:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f267a000640 1 -- 172.21.15.46:0/3366939646 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649e6c88a40 con 0x5649e6d2b190 2024-04-17T08:02:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f267a000640 10 client.16095 handle_command_reply: tid=0 2024-04-17T08:02:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 client.16095 shutdown 2024-04-17T08:02:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 msgr2=0x5649e6d1fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d1fc70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f26740ba680 tx=0x7f26740f57f0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 shutdown_connections 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d2b190 0x5649e6d2d610 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d26cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d1fc70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 --2- 172.21.15.46:0/3366939646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d1f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 >> 172.21.15.46:0/3366939646 conn(0x5649e6d106d0 msgr2=0x5649e6d0f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f267a000640 0 client.16095 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:02:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 shutdown_connections 2024-04-17T08:02:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 1 -- 172.21.15.46:0/3366939646 wait complete. 2024-04-17T08:02:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.246+0000 7f269640e740 20 client.16095 trim_cache size 0 max 16384 2024-04-17T08:02:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.261+0000 7f269640e740 1 Processor -- start 2024-04-17T08:02:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f269640e740 1 -- start start 2024-04-17T08:02:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f269640e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d14040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f269640e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d26c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f269640e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d31f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f269640e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1af4a0 con 0x5649e6d15b50 2024-04-17T08:02:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f269640e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1acd00 con 0x5649e6d1acc0 2024-04-17T08:02:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f269640e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1aab50 con 0x5649e6d15160 2024-04-17T08:02:11.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f2683e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d26c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f2690c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d14040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f2683e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d26c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44112/0 (socket says 172.21.15.46:44112) 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f2690c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d14040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:43416/0 (socket says 172.21.15.46:43416) 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f2683e00640 1 -- 172.21.15.46:0/3683798989 learned_addr learned my addr 172.21.15.46:0/3683798989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.262+0000 7f2691600640 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d31f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2683e00640 1 -- 172.21.15.46:0/3683798989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 msgr2=0x5649e6d14040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2683e00640 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d14040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2683e00640 1 -- 172.21.15.46:0/3683798989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 msgr2=0x5649e6d31f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2683e00640 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d31f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2683e00640 1 -- 172.21.15.46:0/3683798989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e698d3f0 con 0x5649e6d15b50 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2690c00640 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d14040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2683e00640 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d26c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26740be1b0 tx=0x7f26740f5f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc519d73d32d419a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2681200640 1 -- 172.21.15.46:0/3683798989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267404b070 con 0x5649e6d15b50 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2681200640 1 -- 172.21.15.46:0/3683798989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26740efe50 con 0x5649e6d15b50 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.263+0000 7f2681200640 1 -- 172.21.15.46:0/3683798989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740bdb80 con 0x5649e6d15b50 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 -- 172.21.15.46:0/3683798989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 msgr2=0x5649e6d26c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d26c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f26740be1b0 tx=0x7f26740f5f30 comp rx=0 tx=0).stop 2024-04-17T08:02:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 -- 172.21.15.46:0/3683798989 shutdown_connections 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d31f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15b50 0x5649e6d26c70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 --2- 172.21.15.46:0/3683798989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15160 0x5649e6d14040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 -- 172.21.15.46:0/3683798989 >> 172.21.15.46:0/3683798989 conn(0x5649e6d106d0 msgr2=0x5649e6d13930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 -- 172.21.15.46:0/3683798989 shutdown_connections 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.264+0000 7f269640e740 1 -- 172.21.15.46:0/3683798989 wait complete. 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 -- start start 2024-04-17T08:02:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 Processor -- start 2024-04-17T08:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15160 0x5649e6d24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d25760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1ab1c0 con 0x5649e6d15160 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1ac630 con 0x5649e6d1acc0 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1ab890 con 0x5649e6d15b50 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f269640e740 10 client.0 _ll_register_callbacks cb 0x7f26740f59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.265+0000 7f2690c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15160 0x5649e6d24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2683e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15160 0x5649e6d24210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44128/0 (socket says 172.21.15.46:44128) 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 -- 172.21.15.46:0/1922851509 learned_addr learned my addr 172.21.15.46:0/1922851509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2691600640 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d25760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 -- 172.21.15.46:0/1922851509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 msgr2=0x5649e6d24750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d24750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 -- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 msgr2=0x5649e6d25760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d25760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 -- 172.21.15.46:0/1922851509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e6a577d0 con 0x5649e6d15160 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2691600640 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d25760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f2690c00640 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15160 0x5649e6d24210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5649e6d165c0 tx=0x7f267c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4192fa712aa58186 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.266+0000 7f267a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:02:11.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f267a000640 1 -- 172.21.15.46:0/1922851509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c02a7d0 con 0x5649e6d15160 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f267a000640 1 -- 172.21.15.46:0/1922851509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f267c003970 con 0x5649e6d15160 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26740efe50 con 0x5649e6d15160 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f267a000640 1 -- 172.21.15.46:0/1922851509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f267c03b050 con 0x5649e6d15160 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f269640e740 10 client.16101 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649e6d26f50 con 0x5649e6d15160 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f267a000640 1 -- 172.21.15.46:0/1922851509 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26740efe50 con 0x5649e6d15160 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f267a000640 1 -- 172.21.15.46:0/1922851509 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f267c032420 con 0x5649e6d15160 2024-04-17T08:02:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f269640e740 10 client.16101 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.267+0000 7f269640e740 10 client.16101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f269640e740 20 client.16101 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f269640e740 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d91f70 0x5649e6d94390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f269640e740 4 client.16101 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 --> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649e6c828b0 con 0x5649e6d91f70 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f2683e00640 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d91f70 0x5649e6d94390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f2683e00640 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d91f70 0x5649e6d94390 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=796edc04d4a3ed40 server_cookie=757917c3ec2beb66 in_seq=0 out_seq=0 2024-04-17T08:02:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f267a000640 10 client.16101 ms_handle_connect on v2:172.21.15.53:6836/2183592848 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f267a000640 1 -- 172.21.15.46:0/1922851509 <== mds.2 v2:172.21.15.53:6836/2183592848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649e6c828b0 con 0x5649e6d91f70 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.268+0000 7f267a000640 10 client.16101 handle_command_reply: tid=0 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 client.16101 shutdown 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15160 msgr2=0x5649e6d24210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15160 0x5649e6d24210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5649e6d165c0 tx=0x7f267c004a40 comp rx=0 tx=0).stop 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 shutdown_connections 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:6836/2183592848,v1:172.21.15.53:6837/2183592848] conn(0x5649e6d91f70 0x5649e6d94390 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649e6d1acc0 0x5649e6d25760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5649e6d15b50 0x5649e6d24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f267a000640 0 client.16101 ms_handle_reset on v2:172.21.15.53:6836/2183592848 2024-04-17T08:02:11.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 --2- 172.21.15.46:0/1922851509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649e6d15160 0x5649e6d24210 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 >> 172.21.15.46:0/1922851509 conn(0x5649e6d106d0 msgr2=0x5649e6d141c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 shutdown_connections 2024-04-17T08:02:11.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 1 -- 172.21.15.46:0/1922851509 wait complete. 2024-04-17T08:02:11.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.269+0000 7f269640e740 20 client.16101 trim_cache size 0 max 16384 2024-04-17T08:02:11.278 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26700a3bb0 msgr2=0x7f26700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26700a3bb0 0x7f26700a6070 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5649e6d15f50 tx=0x7f267c009500 comp rx=0 tx=0).stop 2024-04-17T08:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c102fe0 msgr2=0x7f268c19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c102fe0 0x7f268c19a7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f267402c260 tx=0x7f267402c670 comp rx=0 tx=0).stop 2024-04-17T08:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 shutdown_connections 2024-04-17T08:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f26700a3bb0 0x7f26700a6070 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f268c1039a0 0x7f268c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f268c102fe0 0x7f268c19a7f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 --2- 172.21.15.46:0/3578566635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c101df0 0x7f268c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 >> 172.21.15.46:0/3578566635 conn(0x7f268c0fd360 msgr2=0x7f268c074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 shutdown_connections 2024-04-17T08:02:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.270+0000 7f2692000640 1 -- 172.21.15.46:0/3578566635 wait complete. 2024-04-17T08:02:11.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:11 smithi046 ceph-mon[32038]: pgmap v527: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-04-17T08:02:11.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:11 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2546651248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:02:11.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T08:02:11.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:11 smithi062 ceph-mon[36507]: pgmap v527: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-04-17T08:02:11.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:11 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2546651248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:02:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:11 smithi053 ceph-mon[36437]: pgmap v527: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-04-17T08:02:11.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:11 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2546651248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:02:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 -- 172.21.15.46:0/3891096801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41095d0 msgr2=0x7f81b4109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 --2- 172.21.15.46:0/3891096801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41095d0 0x7f81b4109a50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009ca0 tx=0x7f81a002c100 comp rx=0 tx=0).stop 2024-04-17T08:02:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 -- 172.21.15.46:0/3891096801 shutdown_connections 2024-04-17T08:02:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 --2- 172.21.15.46:0/3891096801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4109f90 0x7f81b4110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 --2- 172.21.15.46:0/3891096801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41095d0 0x7f81b4109a50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 --2- 172.21.15.46:0/3891096801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81b41083e0 0x7f81b4108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 -- 172.21.15.46:0/3891096801 >> 172.21.15.46:0/3891096801 conn(0x7f81b40773a0 msgr2=0x7f81b40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 -- 172.21.15.46:0/3891096801 shutdown_connections 2024-04-17T08:02:11.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.514+0000 7f81b3e00640 1 -- 172.21.15.46:0/3891096801 wait complete. 2024-04-17T08:02:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 Processor -- start 2024-04-17T08:02:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 -- start start 2024-04-17T08:02:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41083e0 0x7f81b419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81b41095d0 0x7f81b419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4109f90 0x7f81b41a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4114070 con 0x7f81b41083e0 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4113ef0 con 0x7f81b4109f90 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.515+0000 7f81b3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41141f0 con 0x7f81b41095d0 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41083e0 0x7f81b419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41083e0 0x7f81b419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44146/0 (socket says 172.21.15.46:44146) 2024-04-17T08:02:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 -- 172.21.15.46:0/1954126964 learned_addr learned my addr 172.21.15.46:0/1954126964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2000640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81b41095d0 0x7f81b419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b3400640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4109f90 0x7f81b41a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 -- 172.21.15.46:0/1954126964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81b41095d0 msgr2=0x7f81b419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81b41095d0 0x7f81b419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 -- 172.21.15.46:0/1954126964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4109f90 msgr2=0x7f81b41a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4109f90 0x7f81b41a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 -- 172.21.15.46:0/1954126964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b41a6350 con 0x7f81b41083e0 2024-04-17T08:02:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.516+0000 7f81b2a00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41083e0 0x7f81b419e780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e9c0 tx=0x7f81a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.517+0000 7f81ab400640 1 -- 172.21.15.46:0/1954126964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a400cd80 con 0x7f81b41083e0 2024-04-17T08:02:11.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.517+0000 7f81ab400640 1 -- 172.21.15.46:0/1954126964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a40035f0 con 0x7f81b41083e0 2024-04-17T08:02:11.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.517+0000 7f81ab400640 1 -- 172.21.15.46:0/1954126964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4010690 con 0x7f81b41083e0 2024-04-17T08:02:11.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.517+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b41a6640 con 0x7f81b41083e0 2024-04-17T08:02:11.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.517+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b41a6b60 con 0x7f81b41083e0 2024-04-17T08:02:11.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.520+0000 7f81bc826740 1 Processor -- start 2024-04-17T08:02:11.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.520+0000 7f81ab400640 1 -- 172.21.15.46:0/1954126964 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81a40037a0 con 0x7f81b41083e0 2024-04-17T08:02:11.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.520+0000 7f81bc826740 1 -- start start 2024-04-17T08:02:11.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.520+0000 7f81bc826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204160 0x55b83c204580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81bc826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c209650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81bc826740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c209cc0 0x55b83c20c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81bc826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b83c16a020 con 0x55b83c204160 2024-04-17T08:02:11.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81bc826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b83c110460 con 0x55b83c204b50 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81bc826740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b83c165fe0 con 0x55b83c209cc0 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204160 0x55b83c204580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204160 0x55b83c204580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44156/0 (socket says 172.21.15.46:44156) 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 -- 172.21.15.46:0/3860744373 learned_addr learned my addr 172.21.15.46:0/3860744373 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b2a00640 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c209650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b2000640 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c209cc0 0x55b83c20c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 -- 172.21.15.46:0/3860744373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c209cc0 msgr2=0x55b83c20c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c209cc0 0x55b83c20c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 -- 172.21.15.46:0/3860744373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 msgr2=0x55b83c209650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c209650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.521+0000 7f81b3400640 1 -- 172.21.15.46:0/3860744373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83c118040 con 0x55b83c204160 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.522+0000 7f81b3400640 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204160 0x55b83c204580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009ca0 tx=0x7f81ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f21f45cc01430d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.522+0000 7f819fe00640 1 -- 172.21.15.46:0/3860744373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac01e070 con 0x55b83c204160 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.522+0000 7f819fe00640 1 -- 172.21.15.46:0/3860744373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81ac009e50 con 0x55b83c204160 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.522+0000 7f819fe00640 1 -- 172.21.15.46:0/3860744373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00eac0 con 0x55b83c204160 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.522+0000 7f81ab400640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81980a3c30 0x7f81980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.522+0000 7f81ab400640 1 -- 172.21.15.46:0/1954126964 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f81a40d0ad0 con 0x7f81b41083e0 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81b2a00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81980a3c30 0x7f81980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 -- 172.21.15.46:0/3860744373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204160 msgr2=0x55b83c204580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204160 0x55b83c204580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009ca0 tx=0x7f81ac011810 comp rx=0 tx=0).stop 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 -- 172.21.15.46:0/3860744373 shutdown_connections 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c209cc0 0x55b83c20c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c209650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 --2- 172.21.15.46:0/3860744373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204160 0x55b83c204580 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 -- 172.21.15.46:0/3860744373 >> 172.21.15.46:0/3860744373 conn(0x55b83c1ff6d0 msgr2=0x55b83c1ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 -- 172.21.15.46:0/3860744373 shutdown_connections 2024-04-17T08:02:11.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81bc826740 1 -- 172.21.15.46:0/3860744373 wait complete. 2024-04-17T08:02:11.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.523+0000 7f81b2a00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81980a3c30 0x7f81980a60f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f81a40a5100 tx=0x7f81a40d5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.524+0000 7f81bc826740 1 -- start start 2024-04-17T08:02:11.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.524+0000 7f81bc826740 1 Processor -- start 2024-04-17T08:02:11.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.524+0000 7f81bc826740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c20e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.524+0000 7f81bc826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204b50 0x55b83c20ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.524+0000 7f81bc826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c209cc0 0x55b83c215bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81bc826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b83c20cc70 con 0x55b83c204b50 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81bc826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b83c123830 con 0x55b83c209cc0 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81bc826740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b83c20cdf0 con 0x55b83c204160 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81bc826740 10 client.0 _ll_register_callbacks cb 0x55b83c0f1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204b50 0x55b83c20ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204b50 0x55b83c20ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44168/0 (socket says 172.21.15.46:44168) 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 -- 172.21.15.46:0/1682857075 learned_addr learned my addr 172.21.15.46:0/1682857075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b3400640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c209cc0 0x55b83c215bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b3400640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c20e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 -- 172.21.15.46:0/1682857075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 msgr2=0x55b83c20e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c20e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 -- 172.21.15.46:0/1682857075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c209cc0 msgr2=0x55b83c215bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c209cc0 0x55b83c215bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b2000640 1 -- 172.21.15.46:0/1682857075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83c127920 con 0x55b83c204b50 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.525+0000 7f81b3400640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c209cc0 0x55b83c215bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.526+0000 7f81b2000640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204b50 0x55b83c20ebb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f81a0000f80 tx=0x7f81a0009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1e4580dc547f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.526+0000 7f819cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.526+0000 7f819cc00640 1 -- 172.21.15.46:0/1682857075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0044070 con 0x55b83c204b50 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.526+0000 7f819cc00640 1 -- 172.21.15.46:0/1682857075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f81a0003970 con 0x55b83c204b50 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.526+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ac009e50 con 0x55b83c204b50 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.526+0000 7f819cc00640 1 -- 172.21.15.46:0/1682857075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81a0051050 con 0x55b83c204b50 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.526+0000 7f81bc826740 10 client.16113 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b83c0f57f0 con 0x55b83c204b50 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f819cc00640 1 -- 172.21.15.46:0/1682857075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81ac009e50 con 0x55b83c204b50 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f819cc00640 1 -- 172.21.15.46:0/1682857075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f81a0034440 con 0x55b83c204b50 2024-04-17T08:02:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f81bc826740 10 client.16113 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f81bc826740 10 client.16113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f81bc826740 20 client.16113 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f81bc826740 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c21a1e0 0x55b83c21c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f81bc826740 4 client.16113 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.527+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b83c177a40 con 0x55b83c21a1e0 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.528+0000 7f81b3400640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c21a1e0 0x55b83c21c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.528+0000 7f81b3400640 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c21a1e0 0x55b83c21c5f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=573d9b733e5c69f9 server_cookie=2a0f4602793e32aa in_seq=0 out_seq=0 2024-04-17T08:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.528+0000 7f819cc00640 10 client.16113 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f819cc00640 1 -- 172.21.15.46:0/1682857075 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b83c177a40 con 0x55b83c21a1e0 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f819cc00640 10 client.16113 handle_command_reply: tid=0 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 client.16113 shutdown 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204b50 msgr2=0x55b83c20ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204b50 0x55b83c20ebb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f81a0000f80 tx=0x7f81a0009960 comp rx=0 tx=0).stop 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 shutdown_connections 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c209cc0 0x55b83c215bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c21a1e0 0x55b83c21c5f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c204b50 0x55b83c20ebb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 --2- 172.21.15.46:0/1682857075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c20e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f819cc00640 0 client.16113 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:02:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 >> 172.21.15.46:0/1682857075 conn(0x55b83c1ff6d0 msgr2=0x55b83c1fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 shutdown_connections 2024-04-17T08:02:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.529+0000 7f81bc826740 1 -- 172.21.15.46:0/1682857075 wait complete. 2024-04-17T08:02:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.530+0000 7f81bc826740 20 client.16113 trim_cache size 0 max 16384 2024-04-17T08:02:11.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.544+0000 7f81bc826740 1 Processor -- start 2024-04-17T08:02:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81bc826740 1 -- start start 2024-04-17T08:02:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81bc826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204160 0x55b83c206010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81bc826740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204b50 0x55b83c206550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81bc826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c215bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81bc826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b3920 con 0x55b83c209cc0 2024-04-17T08:02:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81bc826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b1180 con 0x55b83c204160 2024-04-17T08:02:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81bc826740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41aefd0 con 0x55b83c204b50 2024-04-17T08:02:11.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81b3400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204160 0x55b83c206010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c215bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c215bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44180/0 (socket says 172.21.15.46:44180) 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81b3400640 1 -- 172.21.15.46:0/2210805140 learned_addr learned my addr 172.21.15.46:0/2210805140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.545+0000 7f81b2000640 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204b50 0x55b83c206550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f81b3400640 1 -- 172.21.15.46:0/2210805140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204b50 msgr2=0x55b83c206550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f81b3400640 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204b50 0x55b83c206550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f81b3400640 1 -- 172.21.15.46:0/2210805140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204160 msgr2=0x55b83c206010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f81b3400640 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204160 0x55b83c206010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f81b3400640 1 -- 172.21.15.46:0/2210805140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83be7c3f0 con 0x55b83c209cc0 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f81b3400640 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c215bb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55b83c20db90 tx=0x7f81ac005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cd3b5f7eb8d452 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f819f400640 1 -- 172.21.15.46:0/2210805140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac01e040 con 0x55b83c209cc0 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f819f400640 1 -- 172.21.15.46:0/2210805140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81ac0055a0 con 0x55b83c209cc0 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.546+0000 7f819f400640 1 -- 172.21.15.46:0/2210805140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac003b70 con 0x55b83c209cc0 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 -- 172.21.15.46:0/2210805140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 msgr2=0x55b83c215bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c215bb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55b83c20db90 tx=0x7f81ac005850 comp rx=0 tx=0).stop 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 -- 172.21.15.46:0/2210805140 shutdown_connections 2024-04-17T08:02:11.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c215bb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204b50 0x55b83c206550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 --2- 172.21.15.46:0/2210805140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204160 0x55b83c206010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 -- 172.21.15.46:0/2210805140 >> 172.21.15.46:0/2210805140 conn(0x55b83c1ff6d0 msgr2=0x55b83c1ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 -- 172.21.15.46:0/2210805140 shutdown_connections 2024-04-17T08:02:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.547+0000 7f81bc826740 1 -- 172.21.15.46:0/2210805140 wait complete. 2024-04-17T08:02:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 -- start start 2024-04-17T08:02:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 Processor -- start 2024-04-17T08:02:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c213050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c213590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c2145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41af640 con 0x55b83c209cc0 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b0ab0 con 0x55b83c204b50 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41afd10 con 0x55b83c204160 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.548+0000 7f81bc826740 10 client.0 _ll_register_callbacks cb 0x7f81a0009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c213050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c2145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c2145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44190/0 (socket says 172.21.15.46:44190) 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b3400640 1 -- 172.21.15.46:0/3364573534 learned_addr learned my addr 172.21.15.46:0/3364573534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b2000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c213590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b2000640 1 -- 172.21.15.46:0/3364573534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 msgr2=0x55b83c213050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b2000640 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c213050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b2000640 1 -- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 msgr2=0x55b83c2145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b2000640 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c2145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b2000640 1 -- 172.21.15.46:0/3364573534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83bf467d0 con 0x55b83c204b50 2024-04-17T08:02:11.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b3400640 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c2145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f81b2000640 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c213590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005e30 tx=0x7f81a0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5293a2b7eaf28c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.549+0000 7f819cc00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.550+0000 7f819cc00640 1 -- 172.21.15.46:0/3364573534 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0044050 con 0x55b83c204b50 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.550+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ac0055a0 con 0x55b83c204b50 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.550+0000 7f819cc00640 1 -- 172.21.15.46:0/3364573534 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f81a0034440 con 0x55b83c204b50 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.550+0000 7f819cc00640 1 -- 172.21.15.46:0/3364573534 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81a0063050 con 0x55b83c204b50 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.550+0000 7f81bc826740 10 client.26291 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:11.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.550+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b83c215d30 con 0x55b83c204b50 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.550+0000 7f819cc00640 1 -- 172.21.15.46:0/3364573534 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81ac0055a0 con 0x55b83c204b50 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f819cc00640 1 -- 172.21.15.46:0/3364573534 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f81a0055a10 con 0x55b83c204b50 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f81bc826740 10 client.26291 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f81bc826740 10 client.26291 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f81bc826740 20 client.26291 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f81bc826740 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c280c10 0x55b83c283030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f81bc826740 4 client.26291 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:02:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 --> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b83c1718b0 con 0x55b83c280c10 2024-04-17T08:02:11.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.551+0000 7f81b3400640 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c280c10 0x55b83c283030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f819cc00640 10 client.26291 ms_handle_connect on v2:172.21.15.46:6838/137300158 2024-04-17T08:02:11.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81b3400640 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c280c10 0x55b83c283030 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea4cd6998c0a5757 server_cookie=664502196007175f in_seq=0 out_seq=0 2024-04-17T08:02:11.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f819cc00640 1 -- 172.21.15.46:0/3364573534 <== mds.1 v2:172.21.15.46:6838/137300158 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b83c1718b0 con 0x55b83c280c10 2024-04-17T08:02:11.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f819cc00640 10 client.26291 handle_command_reply: tid=0 2024-04-17T08:02:11.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 client.26291 shutdown 2024-04-17T08:02:11.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 msgr2=0x55b83c213590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c213590 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005e30 tx=0x7f81a0004ab0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 shutdown_connections 2024-04-17T08:02:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:6838/137300158,v1:172.21.15.46:6839/137300158] conn(0x55b83c280c10 0x55b83c283030 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b83c209cc0 0x55b83c2145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b83c204b50 0x55b83c213590 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 --2- 172.21.15.46:0/3364573534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b83c204160 0x55b83c213050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 >> 172.21.15.46:0/3364573534 conn(0x55b83c1ff6d0 msgr2=0x55b83c20e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f819cc00640 0 client.26291 ms_handle_reset on v2:172.21.15.46:6838/137300158 2024-04-17T08:02:11.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 shutdown_connections 2024-04-17T08:02:11.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.552+0000 7f81bc826740 1 -- 172.21.15.46:0/3364573534 wait complete. 2024-04-17T08:02:11.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81bc826740 20 client.26291 trim_cache size 0 max 16384 2024-04-17T08:02:11.562 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:02:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81980a3c30 msgr2=0x7f81980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81980a3c30 0x7f81980a60f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f81a40a5100 tx=0x7f81a40d5040 comp rx=0 tx=0).stop 2024-04-17T08:02:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41083e0 msgr2=0x7f81b419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41083e0 0x7f81b419e780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e9c0 tx=0x7f81a400ee80 comp rx=0 tx=0).stop 2024-04-17T08:02:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 shutdown_connections 2024-04-17T08:02:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f81980a3c30 0x7f81980a60f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4109f90 0x7f81b41a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81b41095d0 0x7f81b419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 --2- 172.21.15.46:0/1954126964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b41083e0 0x7f81b419e780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 >> 172.21.15.46:0/1954126964 conn(0x7f81b40773a0 msgr2=0x7f81b410c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.553+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 shutdown_connections 2024-04-17T08:02:11.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.554+0000 7f81b3e00640 1 -- 172.21.15.46:0/1954126964 wait complete. 2024-04-17T08:02:11.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T08:02:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.795+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3058609246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8001083e0 msgr2=0x7fc800108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.795+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/3058609246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8001083e0 0x7fc800108800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009900 tx=0x7fc7f002bc30 comp rx=0 tx=0).stop 2024-04-17T08:02:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.796+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3058609246 shutdown_connections 2024-04-17T08:02:11.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.796+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/3058609246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc800109f90 0x7fc800110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.796+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/3058609246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001095d0 0x7fc800109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.796+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/3058609246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8001083e0 0x7fc800108800 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.796+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3058609246 >> 172.21.15.46:0/3058609246 conn(0x7fc8000773a0 msgr2=0x7fc8000797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.796+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3058609246 shutdown_connections 2024-04-17T08:02:11.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.796+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3058609246 wait complete. 2024-04-17T08:02:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 Processor -- start 2024-04-17T08:02:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 -- start start 2024-04-17T08:02:11.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001083e0 0x7fc80019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8001095d0 0x7fc80019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800109f90 0x7fc8001a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc800114070 con 0x7fc8001083e0 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc800113ef0 con 0x7fc800109f90 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7ffe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8001141f0 con 0x7fc8001095d0 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.797+0000 7fc7fea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001083e0 0x7fc80019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001083e0 0x7fc80019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44204/0 (socket says 172.21.15.46:44204) 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 -- 172.21.15.46:0/1080829368 learned_addr learned my addr 172.21.15.46:0/1080829368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8001095d0 0x7fc80019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7ff400640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800109f90 0x7fc8001a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 -- 172.21.15.46:0/1080829368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8001095d0 msgr2=0x7fc80019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8001095d0 0x7fc80019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 -- 172.21.15.46:0/1080829368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800109f90 msgr2=0x7fc8001a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800109f90 0x7fc8001a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 -- 172.21.15.46:0/1080829368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8001a6340 con 0x7fc8001083e0 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.798+0000 7fc7fea00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001083e0 0x7fc80019e720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000c00 tx=0x7fc7f002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.799+0000 7fc7f7400640 1 -- 172.21.15.46:0/1080829368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0041070 con 0x7fc8001083e0 2024-04-17T08:02:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.799+0000 7fc7f7400640 1 -- 172.21.15.46:0/1080829368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f0004450 con 0x7fc8001083e0 2024-04-17T08:02:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.799+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8001a65d0 con 0x7fc8001083e0 2024-04-17T08:02:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.799+0000 7fc7f7400640 1 -- 172.21.15.46:0/1080829368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0034440 con 0x7fc8001083e0 2024-04-17T08:02:11.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.799+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8001a6ac0 con 0x7fc8001083e0 2024-04-17T08:02:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.802+0000 7fc80822d740 1 Processor -- start 2024-04-17T08:02:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.802+0000 7fc7f7400640 1 -- 172.21.15.46:0/1080829368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc7f0044070 con 0x7fc8001083e0 2024-04-17T08:02:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.802+0000 7fc80822d740 1 -- start start 2024-04-17T08:02:11.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc80822d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x564533142580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc80822d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533147650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc80822d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x56453314a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc80822d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5645330a8020 con 0x564533142160 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc80822d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56453304e460 con 0x564533147cc0 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc80822d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5645330a3fe0 con 0x564533142b50 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x564533142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x564533142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44206/0 (socket says 172.21.15.46:44206) 2024-04-17T08:02:11.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc7ff400640 1 -- 172.21.15.46:0/2432595016 learned_addr learned my addr 172.21.15.46:0/2432595016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc7fea00640 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.803+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x56453314a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ff400640 1 -- 172.21.15.46:0/2432595016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 msgr2=0x564533147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ff400640 1 -- 172.21.15.46:0/2432595016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 msgr2=0x56453314a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x56453314a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ff400640 1 -- 172.21.15.46:0/2432595016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564533056040 con 0x564533142160 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x564533142580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009fd0 tx=0x7fc7f80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b8e3d9ce0ad463 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7f7400640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7e40a3bb0 0x7fc7e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ef400640 1 -- 172.21.15.46:0/2432595016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8004d20 con 0x564533142160 2024-04-17T08:02:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7f7400640 1 -- 172.21.15.46:0/1080829368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc7f00f22d0 con 0x7fc8001083e0 2024-04-17T08:02:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ef400640 1 -- 172.21.15.46:0/2432595016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f8004050 con 0x564533142160 2024-04-17T08:02:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7ef400640 1 -- 172.21.15.46:0/2432595016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8005260 con 0x564533142160 2024-04-17T08:02:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.804+0000 7fc7fea00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7e40a3bb0 0x7fc7e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc7fea00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7e40a3bb0 0x7fc7e40a6070 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc7f004c000 tx=0x7fc7f00f5d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 -- 172.21.15.46:0/2432595016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 msgr2=0x564533142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x564533142580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009fd0 tx=0x7fc7f80118d0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 -- 172.21.15.46:0/2432595016 shutdown_connections 2024-04-17T08:02:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x56453314a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 --2- 172.21.15.46:0/2432595016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x564533142580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 -- 172.21.15.46:0/2432595016 >> 172.21.15.46:0/2432595016 conn(0x56453313d6d0 msgr2=0x56453313dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 -- 172.21.15.46:0/2432595016 shutdown_connections 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.805+0000 7fc80822d740 1 -- 172.21.15.46:0/2432595016 wait complete. 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.806+0000 7fc80822d740 1 -- start start 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.806+0000 7fc80822d740 1 Processor -- start 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc80822d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142160 0x56453314c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc80822d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142b50 0x56453314cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc80822d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533147cc0 0x564533153c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc80822d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56453314ac70 con 0x564533142b50 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc80822d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564533061830 con 0x564533142160 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc80822d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56453314adf0 con 0x564533147cc0 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc80822d740 10 client.0 _ll_register_callbacks cb 0x56453302ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142b50 0x56453314cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142b50 0x56453314cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44214/0 (socket says 172.21.15.46:44214) 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 -- 172.21.15.46:0/2620634781 learned_addr learned my addr 172.21.15.46:0/2620634781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533147cc0 0x564533153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142160 0x56453314c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 -- 172.21.15.46:0/2620634781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533147cc0 msgr2=0x564533153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533147cc0 0x564533153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 -- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142160 msgr2=0x56453314c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142160 0x56453314c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7f7e00640 1 -- 172.21.15.46:0/2620634781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564533065920 con 0x564533142b50 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.807+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142160 0x56453314c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142b50 0x56453314cc40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8012700 tx=0x7fc7e8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5baeb8f0e96021e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc7e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc7e3e00640 1 -- 172.21.15.46:0/2620634781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800f8c0 con 0x564533142b50 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc7e3e00640 1 -- 172.21.15.46:0/2620634781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc7e800fa60 con 0x564533142b50 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f8004050 con 0x564533142b50 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc7e3e00640 1 -- 172.21.15.46:0/2620634781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7e8025050 con 0x564533142b50 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc80822d740 10 client.16131 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.808+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645330337f0 con 0x564533142b50 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc7e3e00640 1 -- 172.21.15.46:0/2620634781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7f8004050 con 0x564533142b50 2024-04-17T08:02:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc7e3e00640 1 -- 172.21.15.46:0/2620634781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc7e8010430 con 0x564533142b50 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc80822d740 10 client.16131 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc80822d740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc80822d740 20 client.16131 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc80822d740 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331582b0 0x56453315a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc80822d740 4 client.16131 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.809+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645330b5a40 con 0x5645331582b0 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.810+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331582b0 0x56453315a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.810+0000 7fc7ff400640 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331582b0 0x56453315a730 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f32c42bbf110370 server_cookie=16961707d52c1922 in_seq=0 out_seq=0 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.810+0000 7fc7e3e00640 10 client.16131 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc7e3e00640 1 -- 172.21.15.46:0/2620634781 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645330b5a40 con 0x5645331582b0 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc7e3e00640 10 client.16131 handle_command_reply: tid=0 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 client.16131 shutdown 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142b50 msgr2=0x56453314cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142b50 0x56453314cc40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8012700 tx=0x7fc7e8012bc0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 shutdown_connections 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331582b0 0x56453315a730 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533147cc0 0x564533153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142b50 0x56453314cc40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc7e3e00640 0 client.16131 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 --2- 172.21.15.46:0/2620634781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142160 0x56453314c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 >> 172.21.15.46:0/2620634781 conn(0x56453313d6d0 msgr2=0x56453313c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.811+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 shutdown_connections 2024-04-17T08:02:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.812+0000 7fc80822d740 1 -- 172.21.15.46:0/2620634781 wait complete. 2024-04-17T08:02:11.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.812+0000 7fc80822d740 20 client.16131 trim_cache size 0 max 16384 2024-04-17T08:02:11.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.826+0000 7fc80822d740 1 Processor -- start 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc80822d740 1 -- start start 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc80822d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x56453313dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc80822d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533153c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc80822d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x5645331440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc80822d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8001b3980 con 0x564533142160 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc80822d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8001b11e0 con 0x564533147cc0 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc80822d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8001af030 con 0x564533142b50 2024-04-17T08:02:11.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x56453313dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x56453313dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44224/0 (socket says 172.21.15.46:44224) 2024-04-17T08:02:11.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc7ff400640 1 -- 172.21.15.46:0/1543902257 learned_addr learned my addr 172.21.15.46:0/1543902257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.827+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7ff400640 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x5645331440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7f7e00640 1 -- 172.21.15.46:0/1543902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 msgr2=0x5645331440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x5645331440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7f7e00640 1 -- 172.21.15.46:0/1543902257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 msgr2=0x56453313dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x56453313dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7f7e00640 1 -- 172.21.15.46:0/1543902257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564532dba3f0 con 0x564533142b50 2024-04-17T08:02:11.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7ff400640 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x56453313dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:11.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.828+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533153c40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80108a0 tx=0x7fc7e8010430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c6cb1f8567dfa24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc7eea00640 1 -- 172.21.15.46:0/1543902257 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8022040 con 0x564533142b50 2024-04-17T08:02:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc7eea00640 1 -- 172.21.15.46:0/1543902257 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e8010620 con 0x564533142b50 2024-04-17T08:02:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc7eea00640 1 -- 172.21.15.46:0/1543902257 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800fac0 con 0x564533142b50 2024-04-17T08:02:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 -- 172.21.15.46:0/1543902257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 msgr2=0x564533153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533153c40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80108a0 tx=0x7fc7e8010430 comp rx=0 tx=0).stop 2024-04-17T08:02:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 -- 172.21.15.46:0/1543902257 shutdown_connections 2024-04-17T08:02:11.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533147cc0 0x5645331440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142b50 0x564533153c40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 --2- 172.21.15.46:0/1543902257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533142160 0x56453313dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 -- 172.21.15.46:0/1543902257 >> 172.21.15.46:0/1543902257 conn(0x56453313d6d0 msgr2=0x56453314b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 -- 172.21.15.46:0/1543902257 shutdown_connections 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.829+0000 7fc80822d740 1 -- 172.21.15.46:0/1543902257 wait complete. 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 -- start start 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 Processor -- start 2024-04-17T08:02:11.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142160 0x564533150d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142b50 0x564533151260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533147cc0 0x564533152270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8001af6a0 con 0x564533147cc0 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8001b0b10 con 0x564533142b50 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.830+0000 7fc80822d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc8001afd70 con 0x564533142160 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc80822d740 10 client.0 _ll_register_callbacks cb 0x7fc7e8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142160 0x564533150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142b50 0x564533151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142160 0x564533150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:43548/0 (socket says 172.21.15.46:43548) 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7ff400640 1 -- 172.21.15.46:0/56834125 learned_addr learned my addr 172.21.15.46:0/56834125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7ff400640 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533147cc0 0x564533152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7f7e00640 1 -- 172.21.15.46:0/56834125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142160 msgr2=0x564533150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142160 0x564533150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7f7e00640 1 -- 172.21.15.46:0/56834125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533147cc0 msgr2=0x564533152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533147cc0 0x564533152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7f7e00640 1 -- 172.21.15.46:0/56834125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564532e847d0 con 0x564533142b50 2024-04-17T08:02:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.831+0000 7fc7ff400640 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533147cc0 0x564533152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc7f7e00640 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142b50 0x564533151260 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8012c00 tx=0x7fc7e8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73233a4c4f05bdf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc7e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc7e3e00640 1 -- 172.21.15.46:0/56834125 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8010a90 con 0x564533142b50 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc7e3e00640 1 -- 172.21.15.46:0/56834125 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc7e800f800 con 0x564533142b50 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e8010620 con 0x564533142b50 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc7e3e00640 1 -- 172.21.15.46:0/56834125 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7e8005710 con 0x564533142b50 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc80822d740 10 client.26309 fetch_fsmap learned FSMap version 28 2024-04-17T08:02:11.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.832+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564533153990 con 0x564533142b50 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc7e3e00640 1 -- 172.21.15.46:0/56834125 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e8010620 con 0x564533142b50 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc7e3e00640 1 -- 172.21.15.46:0/56834125 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc7e8003c50 con 0x564533142b50 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc80822d740 10 client.26309 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc80822d740 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc80822d740 20 client.26309 populate_metadata read hostname 'smithi046' 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc80822d740 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331bea00 0x5645331c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc80822d740 4 client.26309 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 --> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645330af8b0 con 0x5645331bea00 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.833+0000 7fc7ff400640 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331bea00 0x5645331c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc7ff400640 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331bea00 0x5645331c0e20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d125962fb0bb1f40 server_cookie=74d7b68ca8c66a35 in_seq=0 out_seq=0 2024-04-17T08:02:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc7e3e00640 10 client.26309 ms_handle_connect on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc7e3e00640 1 -- 172.21.15.46:0/56834125 <== mds.0 v2:172.21.15.53:6838/262156089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645330af8b0 con 0x5645331bea00 2024-04-17T08:02:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc7e3e00640 10 client.26309 handle_command_reply: tid=0 2024-04-17T08:02:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 client.26309 shutdown 2024-04-17T08:02:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142b50 msgr2=0x564533151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142b50 0x564533151260 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8012c00 tx=0x7fc7e8005dc0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 shutdown_connections 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:6838/262156089,v1:172.21.15.53:6839/262156089] conn(0x5645331bea00 0x5645331c0e20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564533147cc0 0x564533152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564533142b50 0x564533151260 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 --2- 172.21.15.46:0/56834125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564533142160 0x564533150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc7e3e00640 0 client.26309 ms_handle_reset on v2:172.21.15.53:6838/262156089 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.834+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 >> 172.21.15.46:0/56834125 conn(0x56453313d6d0 msgr2=0x56453313dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.835+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 shutdown_connections 2024-04-17T08:02:11.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.835+0000 7fc80822d740 1 -- 172.21.15.46:0/56834125 wait complete. 2024-04-17T08:02:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.835+0000 7fc80822d740 20 client.26309 trim_cache size 0 max 16384 2024-04-17T08:02:11.843 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-17T08:02:11.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.835+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7e40a3bb0 msgr2=0x7fc7e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.835+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7e40a3bb0 0x7fc7e40a6070 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc7f004c000 tx=0x7fc7f00f5d10 comp rx=0 tx=0).stop 2024-04-17T08:02:11.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.835+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001083e0 msgr2=0x7fc80019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:11.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.835+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001083e0 0x7fc80019e720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000c00 tx=0x7fc7f002c640 comp rx=0 tx=0).stop 2024-04-17T08:02:11.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 shutdown_connections 2024-04-17T08:02:11.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc7e40a3bb0 0x7fc7e40a6070 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc800109f90 0x7fc8001a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8001095d0 0x7fc80019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/1080829368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8001083e0 0x7fc80019e720 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 >> 172.21.15.46:0/1080829368 conn(0x7fc8000773a0 msgr2=0x7fc80010c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 shutdown_connections 2024-04-17T08:02:11.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:11.836+0000 7fc7ffe00640 1 -- 172.21.15.46:0/1080829368 wait complete. 2024-04-17T08:02:12.856 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-17T08:02:12.856 INFO:tasks.fwd_scrub:done joining 2024-04-17T08:02:12.857 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-17T08:02:12.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T08:02:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 -- 172.21.15.46:0/1767125719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 msgr2=0x7f3fd8109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 --2- 172.21.15.46:0/1767125719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd8109c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009900 tx=0x7f3fc002bc30 comp rx=0 tx=0).stop 2024-04-17T08:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 -- 172.21.15.46:0/1767125719 shutdown_connections 2024-04-17T08:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 --2- 172.21.15.46:0/1767125719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd810a190 0x7f3fd8110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 --2- 172.21.15.46:0/1767125719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd8109c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 --2- 172.21.15.46:0/1767125719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fd81085e0 0x7f3fd8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 -- 172.21.15.46:0/1767125719 >> 172.21.15.46:0/1767125719 conn(0x7f3fd80775b0 msgr2=0x7f3fd80799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 -- 172.21.15.46:0/1767125719 shutdown_connections 2024-04-17T08:02:13.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.109+0000 7f3fdda00640 1 -- 172.21.15.46:0/1767125719 wait complete. 2024-04-17T08:02:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.110+0000 7f3fdda00640 1 Processor -- start 2024-04-17T08:02:13.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.110+0000 7f3fdda00640 1 -- start start 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fdda00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81085e0 0x7f3fd819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fdda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fdda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fd810a190 0x7f3fd81a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fdda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd81142c0 con 0x7f3fd81097d0 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fdda00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8114140 con 0x7f3fd81085e0 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fdda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8114440 con 0x7f3fd810a190 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44254/0 (socket says 172.21.15.46:44254) 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 -- 172.21.15.46:0/2093554955 learned_addr learned my addr 172.21.15.46:0/2093554955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81085e0 0x7f3fd819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fd7400640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fd810a190 0x7f3fd81a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 -- 172.21.15.46:0/2093554955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fd810a190 msgr2=0x7f3fd81a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fd810a190 0x7f3fd81a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 -- 172.21.15.46:0/2093554955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81085e0 msgr2=0x7f3fd819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81085e0 0x7f3fd819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fce000640 1 -- 172.21.15.46:0/2093554955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd81a64f0 con 0x7f3fd81097d0 2024-04-17T08:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.111+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81085e0 0x7f3fd819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.112+0000 7f3fce000640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd819ee10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0000c00 tx=0x7f3fc002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.112+0000 7f3fcfe00640 1 -- 172.21.15.46:0/2093554955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0043020 con 0x7f3fd81097d0 2024-04-17T08:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.112+0000 7f3fcfe00640 1 -- 172.21.15.46:0/2093554955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc00054e0 con 0x7f3fd81097d0 2024-04-17T08:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.112+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd81a6780 con 0x7f3fd81097d0 2024-04-17T08:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.112+0000 7f3fcfe00640 1 -- 172.21.15.46:0/2093554955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0003c40 con 0x7f3fd81097d0 2024-04-17T08:02:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.112+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd81a6d20 con 0x7f3fd81097d0 2024-04-17T08:02:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.113+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd810f900 con 0x7f3fd81097d0 2024-04-17T08:02:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.115+0000 7f3fcfe00640 1 -- 172.21.15.46:0/2093554955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3fc0046020 con 0x7f3fd81097d0 2024-04-17T08:02:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.117+0000 7f3fcfe00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3fbc0a3c30 0x7f3fbc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.117+0000 7f3fcfe00640 1 -- 172.21.15.46:0/2093554955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f3fc00f27a0 con 0x7f3fd81097d0 2024-04-17T08:02:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.117+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3fbc0a3c30 0x7f3fbc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.118+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3fbc0a3c30 0x7f3fbc0a60f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8109630 tx=0x7f3fc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.122+0000 7f3fcfe00640 1 -- 172.21.15.46:0/2093554955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3fc004d020 con 0x7f3fd81097d0 2024-04-17T08:02:13.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:13 smithi062 ceph-mon[36507]: pgmap v528: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 13 op/s 2024-04-17T08:02:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.332+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fd8062b60 con 0x7f3fd81097d0 2024-04-17T08:02:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.335+0000 7f3fcfe00640 1 -- 172.21.15.46:0/2093554955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7f3fc00bed90 con 0x7f3fd81097d0 2024-04-17T08:02:13.336 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:02:13.336 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T08:02:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3fbc0a3c30 msgr2=0x7f3fbc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3fbc0a3c30 0x7f3fbc0a60f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8109630 tx=0x7f3fc8005f90 comp rx=0 tx=0).stop 2024-04-17T08:02:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 msgr2=0x7f3fd819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd819ee10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0000c00 tx=0x7f3fc002c650 comp rx=0 tx=0).stop 2024-04-17T08:02:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 shutdown_connections 2024-04-17T08:02:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f3fbc0a3c30 0x7f3fbc0a60f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fd810a190 0x7f3fd81a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd81097d0 0x7f3fd819ee10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 --2- 172.21.15.46:0/2093554955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81085e0 0x7f3fd819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.338+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 >> 172.21.15.46:0/2093554955 conn(0x7f3fd80775b0 msgr2=0x7f3fd810cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.339+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 shutdown_connections 2024-04-17T08:02:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.339+0000 7f3fdda00640 1 -- 172.21.15.46:0/2093554955 wait complete. 2024-04-17T08:02:13.355 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:13 smithi046 ceph-mon[32038]: pgmap v528: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 13 op/s 2024-04-17T08:02:13.360 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T08:02:13.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T08:02:13.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:13 smithi053 ceph-mon[36437]: pgmap v528: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 13 op/s 2024-04-17T08:02:13.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 -- 172.21.15.46:0/3084134108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970102820 msgr2=0x7f6970104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 --2- 172.21.15.46:0/3084134108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970102820 0x7f6970104c10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6960009ca0 tx=0x7f696002c100 comp rx=0 tx=0).stop 2024-04-17T08:02:13.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 -- 172.21.15.46:0/3084134108 shutdown_connections 2024-04-17T08:02:13.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 --2- 172.21.15.46:0/3084134108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6970107a80 0x7f6970109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 --2- 172.21.15.46:0/3084134108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 0x7f6970107540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 --2- 172.21.15.46:0/3084134108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970102820 0x7f6970104c10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 -- 172.21.15.46:0/3084134108 >> 172.21.15.46:0/3084134108 conn(0x7f69700fc230 msgr2=0x7f69700fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 -- 172.21.15.46:0/3084134108 shutdown_connections 2024-04-17T08:02:13.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.583+0000 7f696fe00640 1 -- 172.21.15.46:0/3084134108 wait complete. 2024-04-17T08:02:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 Processor -- start 2024-04-17T08:02:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 -- start start 2024-04-17T08:02:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6970102820 0x7f697019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 0x7f697019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970107a80 0x7f69701a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f697010da20 con 0x7f6970105150 2024-04-17T08:02:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f697010d8a0 con 0x7f6970107a80 2024-04-17T08:02:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.584+0000 7f696fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f697010dba0 con 0x7f6970102820 2024-04-17T08:02:13.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 0x7f697019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 0x7f697019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44280/0 (socket says 172.21.15.46:44280) 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 -- 172.21.15.46:0/1695560524 learned_addr learned my addr 172.21.15.46:0/1695560524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696f400640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970107a80 0x7f69701a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696ea00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6970102820 0x7f697019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 -- 172.21.15.46:0/1695560524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6970102820 msgr2=0x7f697019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6970102820 0x7f697019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 -- 172.21.15.46:0/1695560524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970107a80 msgr2=0x7f69701a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970107a80 0x7f69701a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.585+0000 7f696e000640 1 -- 172.21.15.46:0/1695560524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69701a1ed0 con 0x7f6970105150 2024-04-17T08:02:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.586+0000 7f696e000640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 0x7f697019a7f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f695c00e9c0 tx=0x7f695c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.586+0000 7f6967400640 1 -- 172.21.15.46:0/1695560524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c00cd80 con 0x7f6970105150 2024-04-17T08:02:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.586+0000 7f6967400640 1 -- 172.21.15.46:0/1695560524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c005500 con 0x7f6970105150 2024-04-17T08:02:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.586+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69701a21c0 con 0x7f6970105150 2024-04-17T08:02:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.586+0000 7f6967400640 1 -- 172.21.15.46:0/1695560524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c010690 con 0x7f6970105150 2024-04-17T08:02:13.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.586+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69701a2710 con 0x7f6970105150 2024-04-17T08:02:13.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.587+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69701095d0 con 0x7f6970105150 2024-04-17T08:02:13.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.590+0000 7f6967400640 1 -- 172.21.15.46:0/1695560524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f695c0046c0 con 0x7f6970105150 2024-04-17T08:02:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.591+0000 7f6967400640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f69540a3ba0 0x7f69540a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.591+0000 7f6967400640 1 -- 172.21.15.46:0/1695560524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f695c0d0b50 con 0x7f6970105150 2024-04-17T08:02:13.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.592+0000 7f696ea00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f69540a3ba0 0x7f69540a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:13.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.592+0000 7f696ea00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f69540a3ba0 0x7f69540a6060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f696002c710 tx=0x7f69600323c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:13.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.595+0000 7f6967400640 1 -- 172.21.15.46:0/1695560524 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f695c09d140 con 0x7f6970105150 2024-04-17T08:02:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.803+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6970062b60 con 0x7f6970105150 2024-04-17T08:02:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.805+0000 7f6967400640 1 -- 172.21.15.46:0/1695560524 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f695c09ca40 con 0x7f6970105150 2024-04-17T08:02:13.805 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T08:02:13.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f69540a3ba0 msgr2=0x7f69540a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f69540a3ba0 0x7f69540a6060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f696002c710 tx=0x7f69600323c0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 msgr2=0x7f697019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:13.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 0x7f697019a7f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f695c00e9c0 tx=0x7f695c00ee80 comp rx=0 tx=0).stop 2024-04-17T08:02:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 shutdown_connections 2024-04-17T08:02:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f69540a3ba0 0x7f69540a6060 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6970107a80 0x7f69701a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6970105150 0x7f697019a7f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 --2- 172.21.15.46:0/1695560524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6970102820 0x7f697019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 >> 172.21.15.46:0/1695560524 conn(0x7f69700fc230 msgr2=0x7f6970101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 shutdown_connections 2024-04-17T08:02:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:13.807+0000 7f696fe00640 1 -- 172.21.15.46:0/1695560524 wait complete. 2024-04-17T08:02:13.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T08:02:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 -- 172.21.15.46:0/2056551201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 msgr2=0x7f24b410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 --2- 172.21.15.46:0/2056551201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 0x7f24b410a210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009ca0 tx=0x7f24ac02c100 comp rx=0 tx=0).stop 2024-04-17T08:02:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 -- 172.21.15.46:0/2056551201 shutdown_connections 2024-04-17T08:02:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 --2- 172.21.15.46:0/2056551201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 0x7f24b410a210 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 --2- 172.21.15.46:0/2056551201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4102fc0 0x7f24b4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 --2- 172.21.15.46:0/2056551201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24b4101dd0 0x7f24b41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 -- 172.21.15.46:0/2056551201 >> 172.21.15.46:0/2056551201 conn(0x7f24b40fd340 msgr2=0x7f24b40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:14.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 -- 172.21.15.46:0/2056551201 shutdown_connections 2024-04-17T08:02:14.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.043+0000 7f24bae00640 1 -- 172.21.15.46:0/2056551201 wait complete. 2024-04-17T08:02:14.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.044+0000 7f24bae00640 1 Processor -- start 2024-04-17T08:02:14.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.044+0000 7f24bae00640 1 -- start start 2024-04-17T08:02:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24bae00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4101dd0 0x7f24b419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24bae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24b4102fc0 0x7f24b419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24bae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 0x7f24b41a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24bae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f24b410da30 con 0x7f24b4103980 2024-04-17T08:02:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24bae00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24b410d8b0 con 0x7f24b4101dd0 2024-04-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24bae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f24b410dbb0 con 0x7f24b4102fc0 2024-04-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24b9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4101dd0 0x7f24b419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24b9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24b4102fc0 0x7f24b419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24b9a00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4101dd0 0x7f24b419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:47492/0 (socket says 172.21.15.46:47492) 2024-04-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24b9a00640 1 -- 172.21.15.46:0/2810101781 learned_addr learned my addr 172.21.15.46:0/2810101781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.045+0000 7f24ba400640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 0x7f24b41a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.046+0000 7f24b9000640 1 -- 172.21.15.46:0/2810101781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4101dd0 msgr2=0x7f24b419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.046+0000 7f24b9000640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4101dd0 0x7f24b419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.046+0000 7f24b9000640 1 -- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 msgr2=0x7f24b41a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.046+0000 7f24b9000640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 0x7f24b41a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.046+0000 7f24b9000640 1 -- 172.21.15.46:0/2810101781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b41a1ed0 con 0x7f24b4102fc0 2024-04-17T08:02:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.046+0000 7f24b9a00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4101dd0 0x7f24b419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.046+0000 7f24b9000640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24b4102fc0 0x7f24b419a7f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e9f0 tx=0x7f24a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.047+0000 7f24b2000640 1 -- 172.21.15.46:0/2810101781 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a000ccc0 con 0x7f24b4102fc0 2024-04-17T08:02:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.047+0000 7f24b2000640 1 -- 172.21.15.46:0/2810101781 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a0010910 con 0x7f24b4102fc0 2024-04-17T08:02:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.047+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b41a21c0 con 0x7f24b4102fc0 2024-04-17T08:02:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.047+0000 7f24b2000640 1 -- 172.21.15.46:0/2810101781 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0018960 con 0x7f24b4102fc0 2024-04-17T08:02:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.047+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b41a2760 con 0x7f24b4102fc0 2024-04-17T08:02:14.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.048+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b41a2a80 con 0x7f24b4102fc0 2024-04-17T08:02:14.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.051+0000 7f24b2000640 1 -- 172.21.15.46:0/2810101781 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f24a0010ac0 con 0x7f24b4102fc0 2024-04-17T08:02:14.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.052+0000 7f24b2000640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f249c0a3c30 0x7f249c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.052+0000 7f24b2000640 1 -- 172.21.15.46:0/2810101781 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f24a00d0a90 con 0x7f24b4102fc0 2024-04-17T08:02:14.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.053+0000 7f24b9a00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f249c0a3c30 0x7f249c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.053+0000 7f24b9a00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f249c0a3c30 0x7f249c0a60f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24a80058c0 tx=0x7f24a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.057+0000 7f24b2000640 1 -- 172.21.15.46:0/2810101781 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f24a009ca90 con 0x7f24b4102fc0 2024-04-17T08:02:14.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:14 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2093554955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:02:14.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:14 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1695560524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T08:02:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.265+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24b41a2e10 con 0x7f24b4102fc0 2024-04-17T08:02:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.266+0000 7f24b2000640 1 -- 172.21.15.46:0/2810101781 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f24a00a0d90 con 0x7f24b4102fc0 2024-04-17T08:02:14.266 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T08:02:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f249c0a3c30 msgr2=0x7f249c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f249c0a3c30 0x7f249c0a60f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24a80058c0 tx=0x7f24a8005f90 comp rx=0 tx=0).stop 2024-04-17T08:02:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24b4102fc0 msgr2=0x7f24b419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24b4102fc0 0x7f24b419a7f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e9f0 tx=0x7f24a000eeb0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 shutdown_connections 2024-04-17T08:02:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f249c0a3c30 0x7f249c0a60f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4103980 0x7f24b41a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24b4102fc0 0x7f24b419a7f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 --2- 172.21.15.46:0/2810101781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b4101dd0 0x7f24b419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 >> 172.21.15.46:0/2810101781 conn(0x7f24b40fd340 msgr2=0x7f24b4074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:14.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 shutdown_connections 2024-04-17T08:02:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.269+0000 7f24bae00640 1 -- 172.21.15.46:0/2810101781 wait complete. 2024-04-17T08:02:14.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T08:02:14.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:14 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2093554955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:02:14.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:14 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1695560524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T08:02:14.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:14 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2093554955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:02:14.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:14 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1695560524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T08:02:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 -- 172.21.15.46:0/2785459624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 msgr2=0x7f7e08110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 --2- 172.21.15.46:0/2785459624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 0x7f7e08110800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009be0 tx=0x7f7e0002bef0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 -- 172.21.15.46:0/2785459624 shutdown_connections 2024-04-17T08:02:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 --2- 172.21.15.46:0/2785459624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 0x7f7e08110800 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 --2- 172.21.15.46:0/2785459624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081095b0 0x7f7e08109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 --2- 172.21.15.46:0/2785459624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e081083c0 0x7f7e081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 -- 172.21.15.46:0/2785459624 >> 172.21.15.46:0/2785459624 conn(0x7f7e080773a0 msgr2=0x7f7e080797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 -- 172.21.15.46:0/2785459624 shutdown_connections 2024-04-17T08:02:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.522+0000 7f7e0da00640 1 -- 172.21.15.46:0/2785459624 wait complete. 2024-04-17T08:02:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.523+0000 7f7e0da00640 1 Processor -- start 2024-04-17T08:02:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.523+0000 7f7e0da00640 1 -- start start 2024-04-17T08:02:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e0da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081083c0 0x7f7e0819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e0da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e081095b0 0x7f7e0819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e0da00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 0x7f7e081a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e0da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08114020 con 0x7f7e081083c0 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e0da00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08113ea0 con 0x7f7e08109f70 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e0da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e081141a0 con 0x7f7e081095b0 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081083c0 0x7f7e0819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081083c0 0x7f7e0819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44330/0 (socket says 172.21.15.46:44330) 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e07e00640 1 -- 172.21.15.46:0/830990590 learned_addr learned my addr 172.21.15.46:0/830990590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e07400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e081095b0 0x7f7e0819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.524+0000 7f7e0d000640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 0x7f7e081a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e07e00640 1 -- 172.21.15.46:0/830990590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e081095b0 msgr2=0x7f7e0819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e07e00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e081095b0 0x7f7e0819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e07e00640 1 -- 172.21.15.46:0/830990590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 msgr2=0x7f7e081a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e07e00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 0x7f7e081a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e07e00640 1 -- 172.21.15.46:0/830990590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e081a6220 con 0x7f7e081083c0 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e07400640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e081095b0 0x7f7e0819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e07e00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081083c0 0x7f7e0819e600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7df800e9f0 tx=0x7f7df800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e04c00640 1 -- 172.21.15.46:0/830990590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df800ccc0 con 0x7f7e081083c0 2024-04-17T08:02:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e04c00640 1 -- 172.21.15.46:0/830990590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df80035f0 con 0x7f7e081083c0 2024-04-17T08:02:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e04c00640 1 -- 172.21.15.46:0/830990590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df800ce60 con 0x7f7e081083c0 2024-04-17T08:02:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.525+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e081a6510 con 0x7f7e081083c0 2024-04-17T08:02:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.526+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e081a6ab0 con 0x7f7e081083c0 2024-04-17T08:02:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.527+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0810f680 con 0x7f7e081083c0 2024-04-17T08:02:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.529+0000 7f7e04c00640 1 -- 172.21.15.46:0/830990590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7df80037a0 con 0x7f7e081083c0 2024-04-17T08:02:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.530+0000 7f7e04c00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7dec0a3c20 0x7f7dec0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.530+0000 7f7e04c00640 1 -- 172.21.15.46:0/830990590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7df80d0a30 con 0x7f7e081083c0 2024-04-17T08:02:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.531+0000 7f7e07400640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7dec0a3c20 0x7f7dec0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.531+0000 7f7e07400640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7dec0a3c20 0x7f7dec0a60e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7e0819fb10 tx=0x7f7df4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.533+0000 7f7e04c00640 1 -- 172.21.15.46:0/830990590 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7df809d020 con 0x7f7e081083c0 2024-04-17T08:02:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.740+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e0802d5d0 con 0x7f7e081083c0 2024-04-17T08:02:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.741+0000 7f7e04c00640 1 -- 172.21.15.46:0/830990590 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f7df809c920 con 0x7f7e081083c0 2024-04-17T08:02:14.741 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T08:02:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.744+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7dec0a3c20 msgr2=0x7f7dec0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.744+0000 7f7e0da00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7dec0a3c20 0x7f7dec0a60e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7e0819fb10 tx=0x7f7df4005f90 comp rx=0 tx=0).stop 2024-04-17T08:02:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.744+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081083c0 msgr2=0x7f7e0819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.744+0000 7f7e0da00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081083c0 0x7f7e0819e600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7df800e9f0 tx=0x7f7df800eeb0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.744+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 shutdown_connections 2024-04-17T08:02:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.745+0000 7f7e0da00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f7dec0a3c20 0x7f7dec0a60e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.745+0000 7f7e0da00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e08109f70 0x7f7e081a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.745+0000 7f7e0da00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e081095b0 0x7f7e0819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.745+0000 7f7e0da00640 1 --2- 172.21.15.46:0/830990590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e081083c0 0x7f7e0819e600 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.745+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 >> 172.21.15.46:0/830990590 conn(0x7f7e080773a0 msgr2=0x7f7e0810c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.745+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 shutdown_connections 2024-04-17T08:02:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.745+0000 7f7e0da00640 1 -- 172.21.15.46:0/830990590 wait complete. 2024-04-17T08:02:14.765 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T08:02:14.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 -- 172.21.15.46:0/1474769161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 msgr2=0x7f65dc105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1474769161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 0x7f65dc105650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009900 tx=0x7f65c402bc30 comp rx=0 tx=0).stop 2024-04-17T08:02:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 -- 172.21.15.46:0/1474769161 shutdown_connections 2024-04-17T08:02:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1474769161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc105b90 0x7f65dc10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1474769161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 0x7f65dc105650 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1474769161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 0x7f65dc104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 -- 172.21.15.46:0/1474769161 >> 172.21.15.46:0/1474769161 conn(0x7f65dc0ff570 msgr2=0x7f65dc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 -- 172.21.15.46:0/1474769161 shutdown_connections 2024-04-17T08:02:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.994+0000 7f65dbe00640 1 -- 172.21.15.46:0/1474769161 wait complete. 2024-04-17T08:02:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.995+0000 7f65dbe00640 1 Processor -- start 2024-04-17T08:02:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.995+0000 7f65dbe00640 1 -- start start 2024-04-17T08:02:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 0x7f65dc19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65dbe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 0x7f65dc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65dbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc105b90 0x7f65dc1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65dbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc10fcc0 con 0x7f65dc103fe0 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65dbe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc10fb40 con 0x7f65dc1051d0 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65dbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc10fe40 con 0x7f65dc105b90 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 0x7f65dc19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 0x7f65dc19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44352/0 (socket says 172.21.15.46:44352) 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65daa00640 1 -- 172.21.15.46:0/1656711860 learned_addr learned my addr 172.21.15.46:0/1656711860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65da000640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 0x7f65dc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.996+0000 7f65db400640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc105b90 0x7f65dc1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65daa00640 1 -- 172.21.15.46:0/1656711860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc105b90 msgr2=0x7f65dc1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65daa00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc105b90 0x7f65dc1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65daa00640 1 -- 172.21.15.46:0/1656711860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 msgr2=0x7f65dc19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65daa00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 0x7f65dc19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65daa00640 1 -- 172.21.15.46:0/1656711860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc1a1f40 con 0x7f65dc103fe0 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65db400640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc105b90 0x7f65dc1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65daa00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 0x7f65dc19a320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00e9c0 tx=0x7f65cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65d3400640 1 -- 172.21.15.46:0/1656711860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00cd80 con 0x7f65dc103fe0 2024-04-17T08:02:15.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65d3400640 1 -- 172.21.15.46:0/1656711860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65cc0035f0 con 0x7f65dc103fe0 2024-04-17T08:02:15.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65d3400640 1 -- 172.21.15.46:0/1656711860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc010690 con 0x7f65dc103fe0 2024-04-17T08:02:15.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc1a2230 con 0x7f65dc103fe0 2024-04-17T08:02:15.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.997+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a2750 con 0x7f65dc103fe0 2024-04-17T08:02:15.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:14.999+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc10b350 con 0x7f65dc103fe0 2024-04-17T08:02:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.001+0000 7f65d3400640 1 -- 172.21.15.46:0/1656711860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f65cc0037a0 con 0x7f65dc103fe0 2024-04-17T08:02:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.002+0000 7f65d3400640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f65c00a3bb0 0x7f65c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.002+0000 7f65d3400640 1 -- 172.21.15.46:0/1656711860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f65cc0d0ca0 con 0x7f65dc103fe0 2024-04-17T08:02:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.003+0000 7f65da000640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f65c00a3bb0 0x7f65c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.003+0000 7f65da000640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f65c00a3bb0 0x7f65c00a6070 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65c402c240 tx=0x7f65c402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:15.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.007+0000 7f65d3400640 1 -- 172.21.15.46:0/1656711860 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f65cc09d290 con 0x7f65dc103fe0 2024-04-17T08:02:15.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.218+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f65dc062b60 con 0x7f65dc103fe0 2024-04-17T08:02:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.219+0000 7f65d3400640 1 -- 172.21.15.46:0/1656711860 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f65cc09cb90 con 0x7f65dc103fe0 2024-04-17T08:02:15.222 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T08:02:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f65c00a3bb0 msgr2=0x7f65c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f65c00a3bb0 0x7f65c00a6070 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65c402c240 tx=0x7f65c402df90 comp rx=0 tx=0).stop 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 msgr2=0x7f65dc19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 0x7f65dc19a320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00e9c0 tx=0x7f65cc00ee80 comp rx=0 tx=0).stop 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 shutdown_connections 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f65c00a3bb0 0x7f65c00a6070 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc105b90 0x7f65dc1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65dc1051d0 0x7f65dc19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 --2- 172.21.15.46:0/1656711860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65dc103fe0 0x7f65dc19a320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.222+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 >> 172.21.15.46:0/1656711860 conn(0x7f65dc0ff570 msgr2=0x7f65dc108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.223+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 shutdown_connections 2024-04-17T08:02:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.223+0000 7f65dbe00640 1 -- 172.21.15.46:0/1656711860 wait complete. 2024-04-17T08:02:15.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-17T08:02:15.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:15 smithi062 ceph-mon[36507]: pgmap v529: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-04-17T08:02:15.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:15 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2810101781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T08:02:15.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:15 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/830990590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T08:02:15.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:15 smithi046 ceph-mon[32038]: pgmap v529: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-04-17T08:02:15.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:15 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2810101781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T08:02:15.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:15 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/830990590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T08:02:15.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.470+0000 7fc33f200640 1 -- 172.21.15.46:0/3423789322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 msgr2=0x7fc340109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:15.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.470+0000 7fc33f200640 1 --2- 172.21.15.46:0/3423789322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc340109a30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc328009900 tx=0x7fc32802bc40 comp rx=0 tx=0).stop 2024-04-17T08:02:15.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.471+0000 7fc33f200640 1 -- 172.21.15.46:0/3423789322 shutdown_connections 2024-04-17T08:02:15.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.471+0000 7fc33f200640 1 --2- 172.21.15.46:0/3423789322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc340109f70 0x7fc340110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.471+0000 7fc33f200640 1 --2- 172.21.15.46:0/3423789322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc340109a30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.471+0000 7fc33f200640 1 --2- 172.21.15.46:0/3423789322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3401083c0 0x7fc3401087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.471+0000 7fc33f200640 1 -- 172.21.15.46:0/3423789322 >> 172.21.15.46:0/3423789322 conn(0x7fc340077360 msgr2=0x7fc3400797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.471+0000 7fc33f200640 1 -- 172.21.15.46:0/3423789322 shutdown_connections 2024-04-17T08:02:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.471+0000 7fc33f200640 1 -- 172.21.15.46:0/3423789322 wait complete. 2024-04-17T08:02:15.473 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:15 smithi053 ceph-mon[36437]: pgmap v529: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-04-17T08:02:15.473 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:15 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2810101781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T08:02:15.473 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:15 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/830990590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T08:02:15.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 Processor -- start 2024-04-17T08:02:15.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 -- start start 2024-04-17T08:02:15.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3401083c0 0x7fc34019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc34019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc340109f70 0x7fc3401a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc3401140a0 con 0x7fc3401095b0 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc340113f20 con 0x7fc3401083c0 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc33f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc340114220 con 0x7fc340109f70 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.472+0000 7fc337e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc34019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc34019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44360/0 (socket says 172.21.15.46:44360) 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 -- 172.21.15.46:0/1790686179 learned_addr learned my addr 172.21.15.46:0/1790686179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:15.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc33e800640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc340109f70 0x7fc3401a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc33de00640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3401083c0 0x7fc34019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 -- 172.21.15.46:0/1790686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc340109f70 msgr2=0x7fc3401a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc340109f70 0x7fc3401a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 -- 172.21.15.46:0/1790686179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3401083c0 msgr2=0x7fc34019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3401083c0 0x7fc34019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 -- 172.21.15.46:0/1790686179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3401a6200 con 0x7fc3401095b0 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.473+0000 7fc337e00640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc34019eb20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc32802c8f0 tx=0x7fc32802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.474+0000 7fc336a00640 1 -- 172.21.15.46:0/1790686179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328041070 con 0x7fc3401095b0 2024-04-17T08:02:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.474+0000 7fc336a00640 1 -- 172.21.15.46:0/1790686179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc328004450 con 0x7fc3401095b0 2024-04-17T08:02:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.474+0000 7fc336a00640 1 -- 172.21.15.46:0/1790686179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328034440 con 0x7fc3401095b0 2024-04-17T08:02:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.474+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3401a6490 con 0x7fc3401095b0 2024-04-17T08:02:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.474+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3401a69d0 con 0x7fc3401095b0 2024-04-17T08:02:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.475+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34010f610 con 0x7fc3401095b0 2024-04-17T08:02:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.477+0000 7fc336a00640 1 -- 172.21.15.46:0/1790686179 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc328044070 con 0x7fc3401095b0 2024-04-17T08:02:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.478+0000 7fc336a00640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc3240a3cd0 0x7fc3240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.479+0000 7fc336a00640 1 -- 172.21.15.46:0/1790686179 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fc3280f2210 con 0x7fc3401095b0 2024-04-17T08:02:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.479+0000 7fc33de00640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc3240a3cd0 0x7fc3240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.480+0000 7fc33de00640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc3240a3cd0 0x7fc3240a6190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc3300095f0 tx=0x7fc330005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:15.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.483+0000 7fc336a00640 1 -- 172.21.15.46:0/1790686179 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc3280be210 con 0x7fc3401095b0 2024-04-17T08:02:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.745+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc340062b60 con 0x7fc3401095b0 2024-04-17T08:02:15.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.780+0000 7fc336a00640 1 -- 172.21.15.46:0/1790686179 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+745530 (secure 0 0 0) 0x7fc3280c2510 con 0x7fc3401095b0 2024-04-17T08:02:15.785 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:02:15.785 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 18, 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "active_gid": 14152, 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "active_name": "x", 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addrs": { 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4172081855 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4172081855 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.787 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addr": "172.21.15.46:6801/4172081855", 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "active_change": "2024-04-17T07:47:21.351701+0000", 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "active_mgr_features": 4540701547738038271, 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "standbys": [ 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24109, 2024-04-17T08:02:15.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "z", 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:15.789 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.796 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-04-17T08:02:15.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.800 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-04-17T08:02:15.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.806 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.807 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T08:02:15.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.812 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-04-17T08:02:15.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T08:02:15.814 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.816 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T08:02:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.821 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-04-17T08:02:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.835 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.836 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:15.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-04-17T08:02:15.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-04-17T08:02:15.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T08:02:15.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-04-17T08:02:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-04-17T08:02:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:15.849 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.850 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-04-17T08:02:15.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.854 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-04-17T08:02:15.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.860 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.861 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.864 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-04-17T08:02:15.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:15.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-04-17T08:02:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T08:02:15.878 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.879 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.880 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-04-17T08:02:15.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-04-17T08:02:15.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.889 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-04-17T08:02:15.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T08:02:15.893 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-04-17T08:02:15.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T08:02:15.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.901 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-04-17T08:02:15.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.903 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-04-17T08:02:15.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.908 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:15.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.911 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:15.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.917 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:15.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T08:02:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.927 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.928 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.929 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-04-17T08:02:15.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.932 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.933 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-04-17T08:02:15.934 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.935 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.937 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.938 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-04-17T08:02:15.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-04-17T08:02:15.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.942 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.943 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.944 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.945 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.949 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.951 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.954 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.956 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:15.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:15.958 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-04-17T08:02:15.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T08:02:15.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T08:02:15.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.968 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T08:02:15.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.973 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T08:02:15.978 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.979 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.982 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T08:02:15.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.984 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.985 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T08:02:15.986 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.987 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-04-17T08:02:15.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:15.993 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.994 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T08:02:15.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-04-17T08:02:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T08:02:16.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.013 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.015 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T08:02:16.020 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.021 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.023 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T08:02:16.028 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.030 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.030 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T08:02:16.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T08:02:16.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.030 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.038 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T08:02:16.038 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T08:02:16.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.038 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.041 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.041 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.041 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T08:02:16.046 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T08:02:16.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.046 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.046 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:16.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.047 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.048 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-04-17T08:02:16.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.054 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.055 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.056 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.057 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-04-17T08:02:16.067 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:16.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-04-17T08:02:16.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.074 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-04-17T08:02:16.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.078 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.082 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.086 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.088 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-04-17T08:02:16.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T08:02:16.090 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-04-17T08:02:16.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.101 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.103 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.107 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.111 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-04-17T08:02:16.115 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-04-17T08:02:16.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-04-17T08:02:16.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-04-17T08:02:16.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T08:02:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.122 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.123 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.127 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.130 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-04-17T08:02:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.134 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.135 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.139 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.142 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.146 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.151 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.155 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.162 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.164 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.165 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.171 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.172 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.174 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.178 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.180 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-04-17T08:02:16.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.182 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-04-17T08:02:16.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-04-17T08:02:16.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-04-17T08:02:16.188 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.189 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.191 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.192 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.195 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-04-17T08:02:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.200 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.204 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.209 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-04-17T08:02:16.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T08:02:16.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.219 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.220 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.221 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.223 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.224 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-04-17T08:02:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-04-17T08:02:16.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-04-17T08:02:16.227 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.230 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.231 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.235 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.237 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.242 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.246 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.251 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.252 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.253 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.254 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.255 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.256 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-04-17T08:02:16.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.269 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.271 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.275 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-04-17T08:02:16.277 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.279 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.282 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.286 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.288 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:16.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T08:02:16.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.298 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.300 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-04-17T08:02:16.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.304 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.305 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.306 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.308 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-04-17T08:02:16.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.314 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-04-17T08:02:16.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-04-17T08:02:16.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.315 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.320 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.321 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.324 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.325 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.327 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:16.328 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.332 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.333 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.337 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.341 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.347 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.353 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.357 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-04-17T08:02:16.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.363 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.364 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-04-17T08:02:16.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-04-17T08:02:16.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:16.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-04-17T08:02:16.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.385 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.389 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.394 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.396 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.397 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.398 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.401 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.403 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.404 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.407 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.409 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.410 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.411 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-04-17T08:02:16.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.414 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.415 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.417 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.418 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.422 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.423 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-04-17T08:02:16.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-04-17T08:02:16.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.438 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-04-17T08:02:16.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-04-17T08:02:16.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.443 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.446 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.450 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.451 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.454 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.454 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-04-17T08:02:16.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.458 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.459 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.460 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-04-17T08:02:16.461 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.463 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.465 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.466 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.467 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.473 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.476 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.477 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-04-17T08:02:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.482 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.485 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.486 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.488 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:16.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.492 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.493 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.494 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-04-17T08:02:16.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.500 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.503 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.504 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24116, 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "y", 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.512 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.513 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-04-17T08:02:16.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-04-17T08:02:16.520 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.526 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-04-17T08:02:16.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.530 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-04-17T08:02:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T08:02:16.534 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-04-17T08:02:16.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T08:02:16.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.540 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.541 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-04-17T08:02:16.542 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.544 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.545 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.548 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.550 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T08:02:16.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.553 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-04-17T08:02:16.554 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-04-17T08:02:16.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:16.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.558 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:16.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.562 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-04-17T08:02:16.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T08:02:16.565 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.568 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-04-17T08:02:16.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T08:02:16.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.574 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-04-17T08:02:16.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T08:02:16.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-04-17T08:02:16.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-04-17T08:02:16.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-04-17T08:02:16.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.590 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-04-17T08:02:16.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.597 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T08:02:16.598 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.599 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.600 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-04-17T08:02:16.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.609 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-04-17T08:02:16.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.614 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-04-17T08:02:16.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.620 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.623 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-04-17T08:02:16.636 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.638 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.639 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.641 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-04-17T08:02:16.642 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-04-17T08:02:16.643 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-04-17T08:02:16.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-04-17T08:02:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T08:02:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-04-17T08:02:16.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-04-17T08:02:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T08:02:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T08:02:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.648 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-04-17T08:02:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-04-17T08:02:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T08:02:16.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-04-17T08:02:16.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.659 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-04-17T08:02:16.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.669 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-04-17T08:02:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-04-17T08:02:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.670 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-04-17T08:02:16.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T08:02:16.672 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.673 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.675 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.679 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.680 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-04-17T08:02:16.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-04-17T08:02:16.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T08:02:16.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.689 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T08:02:16.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T08:02:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.706 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-04-17T08:02:16.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.720 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T08:02:16.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T08:02:16.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.733 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T08:02:16.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.738 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.739 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T08:02:16.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.743 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.750 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.751 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T08:02:16.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.757 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T08:02:16.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.760 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.761 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.762 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.764 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-04-17T08:02:16.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.775 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:16.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.783 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.784 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.785 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-04-17T08:02:16.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.793 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.801 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.805 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.807 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-04-17T08:02:16.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.810 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.812 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-04-17T08:02:16.814 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.817 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.821 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.823 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.824 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.832 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.833 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-04-17T08:02:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-04-17T08:02:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-04-17T08:02:16.850 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-04-17T08:02:16.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.856 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.864 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.867 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.869 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.873 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.875 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.876 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-04-17T08:02:16.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-04-17T08:02:16.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.881 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.885 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.889 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.893 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.893 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.893 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.895 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.896 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.897 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-04-17T08:02:16.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-04-17T08:02:16.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.909 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.910 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.913 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.914 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.914 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.914 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.914 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.918 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.922 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.927 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.930 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.931 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.933 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-04-17T08:02:16.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T08:02:16.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.938 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.939 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc3240a3cd0 msgr2=0x7fc3240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc3240a3cd0 0x7fc3240a6190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc3300095f0 tx=0x7fc330005f90 comp rx=0 tx=0).stop 2024-04-17T08:02:16.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 msgr2=0x7fc34019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc34019eb20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc32802c8f0 tx=0x7fc32802c660 comp rx=0 tx=0).stop 2024-04-17T08:02:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 shutdown_connections 2024-04-17T08:02:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fc3240a3cd0 0x7fc3240a6190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc340109f70 0x7fc3401a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3401095b0 0x7fc34019eb20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 --2- 172.21.15.46:0/1790686179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3401083c0 0x7fc34019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 >> 172.21.15.46:0/1790686179 conn(0x7fc340077360 msgr2=0x7fc34010c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 shutdown_connections 2024-04-17T08:02:16.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:15.802+0000 7fc33f200640 1 -- 172.21.15.46:0/1790686179 wait complete. 2024-04-17T08:02:16.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:16 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1656711860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T08:02:16.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:16 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1790686179' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T08:02:16.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:16 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1656711860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T08:02:16.943 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:16 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1790686179' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T08:02:16.945 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.945 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.946 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.947 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.948 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.950 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-04-17T08:02:16.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.952 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.955 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.956 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.957 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.962 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.964 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.968 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.969 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.973 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.974 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.975 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.977 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.978 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.979 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-04-17T08:02:16.982 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:16.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.988 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:16.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:16.992 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:16.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:16.998 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:16.999 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.011 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T08:02:17.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.015 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-04-17T08:02:17.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-04-17T08:02:17.020 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-04-17T08:02:17.021 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-04-17T08:02:17.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-04-17T08:02:17.024 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.026 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.030 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.031 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.033 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-04-17T08:02:17.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-04-17T08:02:17.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.040 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.041 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.042 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.043 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.045 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.046 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.046 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.046 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.046 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.054 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.056 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.060 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.066 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.067 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-04-17T08:02:17.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.079 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.083 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.085 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.087 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-04-17T08:02:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.099 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-04-17T08:02:17.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-04-17T08:02:17.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.104 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-04-17T08:02:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.110 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.111 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.112 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.113 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.120 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.123 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.124 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.129 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.130 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.132 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.133 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-04-17T08:02:17.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.139 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.140 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.148 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.149 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.154 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.157 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T08:02:17.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-04-17T08:02:17.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-04-17T08:02:17.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.172 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.175 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.176 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-04-17T08:02:17.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.184 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.185 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.190 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.193 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.195 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.196 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.197 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.199 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.203 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-04-17T08:02:17.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T08:02:17.206 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-04-17T08:02:17.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.215 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.216 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.218 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.219 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.221 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-04-17T08:02:17.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-04-17T08:02:17.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-04-17T08:02:17.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.227 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard", 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-04-17T08:02:17.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.231 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.232 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.233 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.235 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.236 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-04-17T08:02:17.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.245 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.248 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.249 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-04-17T08:02:17.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-04-17T08:02:17.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.252 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-04-17T08:02:17.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:17.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-04-17T08:02:17.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T08:02:17.260 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:17.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.267 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:17.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-04-17T08:02:17.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-04-17T08:02:17.276 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.278 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-04-17T08:02:17.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-04-17T08:02:17.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:17.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T08:02:17.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.283 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-04-17T08:02:17.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T08:02:17.292 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-04-17T08:02:17.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T08:02:17.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.301 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-04-17T08:02:17.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T08:02:17.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-04-17T08:02:17.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.310 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-04-17T08:02:17.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T08:02:17.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.317 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-04-17T08:02:17.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-04-17T08:02:17.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-04-17T08:02:17.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.333 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-04-17T08:02:17.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T08:02:17.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-04-17T08:02:17.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T08:02:17.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.346 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-04-17T08:02:17.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T08:02:17.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.356 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.357 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-04-17T08:02:17.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T08:02:17.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-04-17T08:02:17.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.369 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-04-17T08:02:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T08:02:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.377 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-04-17T08:02:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.384 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-04-17T08:02:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-04-17T08:02:17.387 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.389 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.393 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.395 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.396 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-04-17T08:02:17.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T08:02:17.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.399 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.400 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T08:02:17.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.408 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T08:02:17.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T08:02:17.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T08:02:17.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.423 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T08:02:17.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T08:02:17.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T08:02:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T08:02:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T08:02:17.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.458 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T08:02:17.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T08:02:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.470 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T08:02:17.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T08:02:17.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.470 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.475 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-04-17T08:02:17.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-04-17T08:02:17.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-04-17T08:02:17.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.487 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.491 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.492 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.493 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.494 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.501 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-04-17T08:02:17.502 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.504 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-04-17T08:02:17.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.512 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.513 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.517 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-04-17T08:02:17.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-04-17T08:02:17.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.526 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.530 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.533 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.534 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.539 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:17.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-04-17T08:02:17.545 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-04-17T08:02:17.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T08:02:17.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-04-17T08:02:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-04-17T08:02:17.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.553 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.555 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.557 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-04-17T08:02:17.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-04-17T08:02:17.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.562 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-04-17T08:02:17.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.572 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.576 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.579 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.581 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.584 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.585 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.587 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.588 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.589 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.590 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.593 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.596 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.597 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.600 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.603 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.604 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.609 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-04-17T08:02:17.610 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-04-17T08:02:17.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-04-17T08:02:17.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-04-17T08:02:17.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.618 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.620 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.623 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.624 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.629 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.633 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.634 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.636 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.637 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.638 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.640 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.642 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.649 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.650 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.653 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.656 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.658 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.660 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.664 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-04-17T08:02:17.667 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.669 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.673 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.676 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.678 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.697 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-04-17T08:02:17.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.701 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.702 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.704 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-04-17T08:02:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T08:02:17.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.709 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.710 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.712 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.713 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-04-17T08:02:17.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.719 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-04-17T08:02:17.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-04-17T08:02:17.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.730 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.731 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.732 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.739 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T08:02:17.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.743 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-04-17T08:02:17.744 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.745 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.746 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.750 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.753 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.755 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.759 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.761 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.765 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.766 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-04-17T08:02:17.768 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.770 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.774 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.779 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.786 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.788 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-04-17T08:02:17.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.790 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-04-17T08:02:17.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.796 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.798 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-04-17T08:02:17.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-04-17T08:02:17.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-04-17T08:02:17.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.800 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-04-17T08:02:17.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.807 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.808 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.810 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.813 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.820 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.821 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.823 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.824 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.828 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.829 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.833 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.834 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.835 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.837 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-04-17T08:02:17.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.841 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.843 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.850 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.855 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.858 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-04-17T08:02:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-04-17T08:02:17.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-04-17T08:02:17.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.869 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.870 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-04-17T08:02:17.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-04-17T08:02:17.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.873 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.878 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.882 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.882 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.883 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-04-17T08:02:17.885 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.889 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.890 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.892 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.893 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.894 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.899 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.901 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.903 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-04-17T08:02:17.906 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T08:02:17.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T08:02:17.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-04-17T08:02:17.913 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-04-17T08:02:17.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.920 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.924 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-04-17T08:02:17.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.929 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-04-17T08:02:17.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-04-17T08:02:17.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard": "https://172.21.15.46:8443/" 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: "always_on_modules": { 2024-04-17T08:02:17.932 INFO:teuthology.orchestra.run.smithi046.stdout: "octopus": [ 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-04-17T08:02:17.933 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "pacific": [ 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-04-17T08:02:17.934 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "quincy": [ 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-04-17T08:02:17.935 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: "reef": [ 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-04-17T08:02:17.937 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: "squid": [ 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-04-17T08:02:17.938 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.939 INFO:teuthology.orchestra.run.smithi046.stdout: "last_failure_osd_epoch": 3, 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "active_clients": [ 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-04-17T08:02:17.940 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 177422763 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "libcephsqlite", 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-04-17T08:02:17.941 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-04-17T08:02:17.942 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2597234954 2024-04-17T08:02:17.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.942 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:17.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-04-17T08:02:17.942 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3467161945 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2345998964 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:17.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:17.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-17T08:02:17.945 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:02:17.953 DEBUG:tasks.check_counter:Getting stats from a 2024-04-17T08:02:17.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-17T08:02:17.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:16 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1656711860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T08:02:17.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:16 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1790686179' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T08:02:17.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:17 smithi046 ceph-mon[32038]: pgmap v530: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-17T08:02:17.955 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:17 smithi062 ceph-mon[36507]: pgmap v530: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-17T08:02:17.955 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:17 smithi053 ceph-mon[36437]: pgmap v530: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-17T08:02:18.459 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T08:02:19.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:19 smithi046 ceph-mon[32038]: pgmap v531: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-17T08:02:19.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:19 smithi053 ceph-mon[36437]: pgmap v531: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-17T08:02:19.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:19 smithi062 ceph-mon[36507]: pgmap v531: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-04-17T08:02:21.083 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:02:21.084 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:21.084 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-04-17T08:02:21.085 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-04-17T08:02:21.085 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-04-17T08:02:21.085 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-04-17T08:02:21.085 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:21.086 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-04-17T08:02:21.086 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.004477228, 2024-04-17T08:02:21.086 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T08:02:21.086 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001045401, 2024-04-17T08:02:21.087 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:21.087 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:21.087 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:21.087 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:21.088 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:21.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.088 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:21.088 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:21.089 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:21.089 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:21.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.089 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T08:02:21.090 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T08:02:21.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.090 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:21.091 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1, 2024-04-17T08:02:21.091 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 88, 2024-04-17T08:02:21.092 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 160, 2024-04-17T08:02:21.092 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 93456, 2024-04-17T08:02:21.092 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 3, 2024-04-17T08:02:21.093 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-04-17T08:02:21.093 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.020150172, 2024-04-17T08:02:21.093 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.013663686, 2024-04-17T08:02:21.093 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.004222777, 2024-04-17T08:02:21.094 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:21.094 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:21.094 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 86, 2024-04-17T08:02:21.094 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.008287912, 2024-04-17T08:02:21.095 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000096371 2024-04-17T08:02:21.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.095 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:21.096 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:21.096 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:21.096 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:21.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.096 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T08:02:21.097 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 93456 2024-04-17T08:02:21.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.097 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:21.097 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 123, 2024-04-17T08:02:21.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 118, 2024-04-17T08:02:21.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 454944, 2024-04-17T08:02:21.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 71376, 2024-04-17T08:02:21.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-04-17T08:02:21.099 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-04-17T08:02:21.099 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.047935064, 2024-04-17T08:02:21.099 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.019371027, 2024-04-17T08:02:21.099 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.029066847, 2024-04-17T08:02:21.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.004892139, 2024-04-17T08:02:21.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:21.100 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 118, 2024-04-17T08:02:21.100 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.013475169, 2024-04-17T08:02:21.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000114196 2024-04-17T08:02:21.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:21.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:21.101 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:21.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:21.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.102 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 454944, 2024-04-17T08:02:21.103 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 71376 2024-04-17T08:02:21.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.103 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-17T08:02:21.103 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-04-17T08:02:21.103 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-17T08:02:21.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.104 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-17T08:02:21.104 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:21.104 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:21.104 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:21.105 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:21.105 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:21.105 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:21.105 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:21.105 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:21.106 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:21.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:21.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:21.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:21.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:21.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:21.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:21.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:21.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:21.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-17T08:02:21.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:21.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:21.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:21.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:21.110 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 149854, 2024-04-17T08:02:21.110 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 28, 2024-04-17T08:02:21.110 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:21.110 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:21.110 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:21.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:21.112 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:21.112 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-17T08:02:21.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-04-17T08:02:21.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-04-17T08:02:21.112 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:21.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-17T08:02:21.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:21.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-17T08:02:21.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:21.113 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:21.113 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 120, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 435160, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 120, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-17T08:02:21.114 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-17T08:02:21.115 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 120, 2024-04-17T08:02:21.115 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 435160, 2024-04-17T08:02:21.115 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:21.115 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:21.115 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:21.115 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:21.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:21.116 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:21.116 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:02:21.364 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:21 smithi046 ceph-mon[32038]: pgmap v532: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T08:02:21.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:21 smithi053 ceph-mon[36437]: pgmap v532: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T08:02:21.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:21 smithi062 ceph-mon[36507]: pgmap v532: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T08:02:22.085 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-17T08:02:22.085 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-17T08:02:22.086 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-17T08:02:22.086 DEBUG:tasks.check_counter:Getting stats from d 2024-04-17T08:02:22.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-17T08:02:22.545 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T08:02:23.261 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:23 smithi046 ceph-mon[32038]: pgmap v533: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T08:02:23.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:23 smithi053 ceph-mon[36437]: pgmap v533: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T08:02:23.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:23 smithi062 ceph-mon[36507]: pgmap v533: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-17T08:02:24.974 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:02:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-04-17T08:02:24.974 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.004277989, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000781426, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:24.975 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:24.976 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 129, 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 124, 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 456576, 2024-04-17T08:02:24.977 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 75216, 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.044653019, 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.018278053, 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.024048631, 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.005061038, 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:24.978 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 124, 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.013367130, 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000107799 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.979 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 456576, 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 75216 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1, 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 94, 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 160, 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 94512, 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 3, 2024-04-17T08:02:24.980 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.020085861, 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.015098945, 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002890888, 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 92, 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.008820836, 2024-04-17T08:02:24.981 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000095878 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T08:02:24.982 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 94512 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:24.983 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:24.984 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:24.985 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:24.986 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-17T08:02:24.987 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148857, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 29, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:24.988 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:24.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-17T08:02:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-04-17T08:02:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-04-17T08:02:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-17T08:02:24.990 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 126, 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 435940, 2024-04-17T08:02:24.991 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 126, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 126, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 435940, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:24.992 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:24.993 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:24.993 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:02:25.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:25 smithi046 ceph-mon[32038]: pgmap v534: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-17T08:02:25.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:25 smithi053 ceph-mon[36437]: pgmap v534: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-17T08:02:25.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:25 smithi062 ceph-mon[36507]: pgmap v534: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-17T08:02:26.071 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-17T08:02:26.071 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-17T08:02:26.071 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-17T08:02:26.071 DEBUG:tasks.check_counter:Getting stats from g 2024-04-17T08:02:26.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-17T08:02:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T08:02:27.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:27 smithi046 ceph-mon[32038]: pgmap v535: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T08:02:27.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:27 smithi053 ceph-mon[36437]: pgmap v535: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T08:02:27.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:27 smithi062 ceph-mon[36507]: pgmap v535: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T08:02:29.094 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:29 smithi046 ceph-mon[32038]: pgmap v536: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-17T08:02:29.094 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:29 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:02:29.261 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:02:29.262 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:29.262 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 5026, 2024-04-17T08:02:29.262 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 8684, 2024-04-17T08:02:29.262 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1591473, 2024-04-17T08:02:29.262 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 11049529, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 30, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 9, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 1.344769070, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.940149401, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.482943957, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.054773962, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 8678, 2024-04-17T08:02:29.263 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.864023669, 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000330032 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5078, 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.003364612, 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000662 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.264 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 736 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2853, 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 5201, 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2959958, 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 1794370, 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 8, 2024-04-17T08:02:29.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 5, 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.668373418, 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.317943223, 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.371820826, 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.032921816, 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5202, 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.829248977, 2024-04-17T08:02:29.266 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000159409 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5937, 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.003982065, 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000670 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-17T08:02:29.267 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 233152 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2577, 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2441, 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1861272, 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 1101462, 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 7, 2024-04-17T08:02:29.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 5, 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.357713403, 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.132414755, 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.199729463, 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.075853134, 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2442, 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.341960585, 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000140032 2024-04-17T08:02:29.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 640, 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000331526, 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000518 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 502432, 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 82016 2024-04-17T08:02:29.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-17T08:02:29.271 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1928, 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.374815485, 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001231750 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 963, 2024-04-17T08:02:29.272 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.222373064, 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000230916 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 1890, 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 1886, 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-04-17T08:02:29.273 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1886, 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 5.224839047, 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002770328 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 4, 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-04-17T08:02:29.274 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 11, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 3521, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 184, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 3337, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 3259, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-04-17T08:02:29.275 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 68, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 68, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 40, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 3797, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 2764, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 4, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 10, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-04-17T08:02:29.276 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 0, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 1084, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 1, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 91, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 3489, 2024-04-17T08:02:29.277 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 2630, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 3452, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 26581383, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 2630, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T08:02:29.278 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 18, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 18, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 2612, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 1, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 68, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 1047, 2024-04-17T08:02:29.279 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 1704, 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 1428, 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 75, 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 1217 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.280 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 143, 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 2, 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-04-17T08:02:29.281 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 607, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 607, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 0, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-04-17T08:02:29.282 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-04-17T08:02:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 4042, 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 4042, 2024-04-17T08:02:29.284 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 6, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 6, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 2, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-04-17T08:02:29.285 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 4194304, 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 11459918, 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 123, 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.083831853, 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.008811641 2024-04-17T08:02:29.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 2402, 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 3009, 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 607, 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 186, 2024-04-17T08:02:29.287 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 187, 2024-04-17T08:02:29.288 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 1, 2024-04-17T08:02:29.288 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 3521, 2024-04-17T08:02:29.288 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 4128, 2024-04-17T08:02:29.288 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 607, 2024-04-17T08:02:29.288 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 68, 2024-04-17T08:02:29.288 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 2284, 2024-04-17T08:02:29.288 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 2216, 2024-04-17T08:02:29.289 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 71564, 2024-04-17T08:02:29.289 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 223608 2024-04-17T08:02:29.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.289 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-04-17T08:02:29.289 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 2929, 2024-04-17T08:02:29.289 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 35, 2024-04-17T08:02:29.289 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 1890, 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 8, 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 1052, 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 375, 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.613297175, 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.004302125 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.290 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4, 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.018237207, 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.004559301 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.291 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-04-17T08:02:29.292 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 897, 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.811699468, 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000904904 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.293 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1, 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000182711, 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000182711 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.296 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.296 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-04-17T08:02:29.296 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.297 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.297 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.297 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-04-17T08:02:29.297 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.299 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.001986984, 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000993492 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-04-17T08:02:29.300 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.302 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.303 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.304 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-04-17T08:02:29.305 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.307 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 607, 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.779435502, 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.004578971 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-04-17T08:02:29.308 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-04-17T08:02:29.309 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 2, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 1, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 1, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 1123, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 1123, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 217, 2024-04-17T08:02:29.310 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:29.311 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:29.312 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:29.313 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:29.314 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 373258, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 2510, 2024-04-17T08:02:29.315 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 880, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 10, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 8888, 2024-04-17T08:02:29.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 170, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 8543478, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 94885, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:29.317 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 4113, 2024-04-17T08:02:29.318 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 7498672, 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 28, 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 4113, 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4113, 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 14.038848563, 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003413286 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.319 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4113, 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 4177 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 4113, 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 2654, 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 1459, 2024-04-17T08:02:29.320 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 0, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 10, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 24, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 147, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 25, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-04-17T08:02:29.321 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 1227, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 2630, 2024-04-17T08:02:29.322 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 20, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T08:02:29.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-04-17T08:02:29.324 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-04-17T08:02:29.325 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-04-17T08:02:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-04-17T08:02:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 93, 2024-04-17T08:02:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-04-17T08:02:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 4, 2024-04-17T08:02:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 12, 2024-04-17T08:02:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 12, 2024-04-17T08:02:29.326 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 79, 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 0, 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 15 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-04-17T08:02:29.327 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-04-17T08:02:29.328 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 16, 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 8, 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 1214, 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 607, 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-17T08:02:29.330 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:29.330 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 10452, 2024-04-17T08:02:29.330 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 5519098, 2024-04-17T08:02:29.330 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:29.330 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 10452, 2024-04-17T08:02:29.330 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-17T08:02:29.330 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 10452, 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 5519098, 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.331 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:29.332 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 4113, 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 7564835, 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 4113, 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 7564835, 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.333 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-17T08:02:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 4113, 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 4113, 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 4113, 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 4113, 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 607, 2024-04-17T08:02:29.336 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 66163, 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 607, 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 24, 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 66163, 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x55e6aaff4460": { 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-17T08:02:29.338 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 4042, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 7265614, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 4042, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 123, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 7265614, 2024-04-17T08:02:29.339 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:29.340 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:29.340 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:29.340 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:29.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:02:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:29 smithi053 ceph-mon[36437]: pgmap v536: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-17T08:02:29.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:29 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:02:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:29 smithi062 ceph-mon[36507]: pgmap v536: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-17T08:02:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:29 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T08:02:30.361 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-17T08:02:30.362 INFO:tasks.check_counter:Daemon mds.g mds.imported=91 2024-04-17T08:02:30.362 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-04-17T08:02:30.362 DEBUG:tasks.check_counter:Getting stats from j 2024-04-17T08:02:30.362 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-17T08:02:30.830 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T08:02:31.301 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:31 smithi046 ceph-mon[32038]: pgmap v537: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T08:02:31.301 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:02:31.301 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:02:31.302 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:31 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:02:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:31 smithi053 ceph-mon[36437]: pgmap v537: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T08:02:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:02:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:02:31.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:31 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:02:31.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:31 smithi062 ceph-mon[36507]: pgmap v537: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T08:02:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T08:02:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T08:02:31.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:31 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' 2024-04-17T08:02:33.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:33 smithi046 ceph-mon[32038]: pgmap v538: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T08:02:33.262 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-17T08:02:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 129, 2024-04-17T08:02:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 126, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 454672, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 77328, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.053095212, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.019131968, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.031481809, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.005880473, 2024-04-17T08:02:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 125, 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.011984699, 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000095877 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 454672, 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 77328 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1, 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 96, 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 160, 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 94864, 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 3, 2024-04-17T08:02:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 2, 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.017667007, 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.013577191, 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001860659, 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 94, 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.008613802, 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000091636 2024-04-17T08:02:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 94864 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2992, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001331184, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000053833, 2024-04-17T08:02:33.268 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001192881, 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.001489540, 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000496513 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:33.269 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-04-17T08:02:33.270 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:33.271 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:33.272 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:33.273 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 149854, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 28, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-04-17T08:02:33.276 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-17T08:02:33.277 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 130, 2024-04-17T08:02:33.278 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 436460, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 130, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 130, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 436460, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-17T08:02:33.279 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-17T08:02:33.280 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-17T08:02:33.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:33.280 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-17T08:02:33.280 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-17T08:02:33.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:33 smithi053 ceph-mon[36437]: pgmap v538: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T08:02:33.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:33 smithi062 ceph-mon[36507]: pgmap v538: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-17T08:02:34.260 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-17T08:02:34.260 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-17T08:02:34.261 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-17T08:02:34.261 DEBUG:tasks.check_counter:Getting stats from b 2024-04-17T08:02:34.261 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-17T08:02:34.527 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T08:02:35.221 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:35 smithi053 ceph-mon[36437]: pgmap v539: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T08:02:35.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:35 smithi062 ceph-mon[36507]: pgmap v539: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T08:02:35.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:35 smithi046 ceph-mon[32038]: pgmap v539: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T08:02:36.485 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-17T08:02:36.485 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:36.485 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 7282, 2024-04-17T08:02:36.485 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 5183, 2024-04-17T08:02:36.485 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1951124, 2024-04-17T08:02:36.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 17727656, 2024-04-17T08:02:36.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 64, 2024-04-17T08:02:36.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-04-17T08:02:36.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.494570690, 2024-04-17T08:02:36.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.201501072, 2024-04-17T08:02:36.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.257215546, 2024-04-17T08:02:36.486 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.077963013, 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5182, 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.826906066, 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000159572 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7196, 2024-04-17T08:02:36.487 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000999632, 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000138 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 18012, 2024-04-17T08:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 36445, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 6023072, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 22827447, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 15, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 1.179880570, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.565786365, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.687287331, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.085037759, 2024-04-17T08:02:36.489 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:36.490 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 36443, 2024-04-17T08:02:36.490 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.546641357, 2024-04-17T08:02:36.490 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000042440 2024-04-17T08:02:36.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.490 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:36.491 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 22622, 2024-04-17T08:02:36.491 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006856454, 2024-04-17T08:02:36.491 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000303 2024-04-17T08:02:36.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 56816, 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 352960 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 7945, 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 5662, 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2692552, 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 14791696, 2024-04-17T08:02:36.492 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 10, 2024-04-17T08:02:36.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-04-17T08:02:36.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.519061477, 2024-04-17T08:02:36.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.195977543, 2024-04-17T08:02:36.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.273768214, 2024-04-17T08:02:36.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.096903003, 2024-04-17T08:02:36.493 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5661, 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.792194674, 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000139938 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6223, 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000868154, 2024-04-17T08:02:36.494 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000139 2024-04-17T08:02:36.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 516192, 2024-04-17T08:02:36.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 89104 2024-04-17T08:02:36.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.495 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-17T08:02:36.495 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-04-17T08:02:36.495 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3009, 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 9.210257944, 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003060903 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3524, 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.664819778, 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000188654 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.498 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-04-17T08:02:36.498 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 9449, 2024-04-17T08:02:36.498 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 9276, 2024-04-17T08:02:36.498 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-04-17T08:02:36.498 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 9276, 2024-04-17T08:02:36.498 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 29.577380090, 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003188592 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 173, 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 202, 2024-04-17T08:02:36.499 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 23, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 777, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 314, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 247, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 216, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 328, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 4373, 2024-04-17T08:02:36.500 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 97, 2024-04-17T08:02:36.501 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 97, 2024-04-17T08:02:36.501 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 52, 2024-04-17T08:02:36.501 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 10422, 2024-04-17T08:02:36.501 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 9771, 2024-04-17T08:02:36.501 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 161, 2024-04-17T08:02:36.501 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 2, 2024-04-17T08:02:36.501 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 2, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 33, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 144, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 4742, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-04-17T08:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 2817, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 3452, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 26581383, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 2817, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-04-17T08:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 200, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 7, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 193, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 2617, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 7, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 136, 2024-04-17T08:02:36.504 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 8475, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 8, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 5105, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 3773, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 453, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 2910 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.505 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 2, 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 130, 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 1, 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-04-17T08:02:36.506 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 2025, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 2022, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 2, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-04-17T08:02:36.507 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 7, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 144, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 23, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-04-17T08:02:36.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 16138, 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 16138, 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-04-17T08:02:36.509 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 21, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 21, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 3, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-04-17T08:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 38550898, 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 641, 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 14.603907203, 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.022783006 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-04-17T08:02:36.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 571, 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 5996, 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 5425, 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 189, 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 285, 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 96, 2024-04-17T08:02:36.512 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 777, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 7094, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 6317, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 97, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 5978, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 5881, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 87916, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 231800 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 9731, 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 2048, 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 9449, 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 161, 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 4057, 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5880, 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 22.241651424, 2024-04-17T08:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003782593 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 73, 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.223542503, 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003062226 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003254935, 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001627467 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 709, 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.129942673, 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001593713 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.517 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000198276, 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000198276 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.518 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-04-17T08:02:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.519 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-04-17T08:02:36.520 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.520 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.520 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.520 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 83, 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.075993217, 2024-04-17T08:02:36.521 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000915580 2024-04-17T08:02:36.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.522 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-04-17T08:02:36.522 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.522 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.522 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.522 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 350, 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.084305683, 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000240873 2024-04-17T08:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 78, 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.650461891, 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.008339255 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 79, 2024-04-17T08:02:36.524 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.154228474, 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001952259 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.525 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 22, 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.460794046, 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.020945183 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-04-17T08:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.527 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10, 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.124929367, 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.012492936 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8, 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002483540, 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000310442 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8, 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.023803313, 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002975414 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-04-17T08:02:36.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7, 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.007540293, 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001077184 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1966, 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4.394250455, 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002235122 2024-04-17T08:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-04-17T08:02:36.532 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 3, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 5031, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 5031, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 237, 2024-04-17T08:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:36.534 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:36.535 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 326457, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 673, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 1232, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 14, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-17T08:02:36.538 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8888, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 170, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:36.539 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 2264271, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 29258, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-04-17T08:02:36.540 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 8483, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 37606518, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 37, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 8483, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8483, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 53.583888201, 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.006316620 2024-04-17T08:02:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8483, 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 8800 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 8483, 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 2879, 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 5604, 2024-04-17T08:02:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 11, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 55, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 689, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 45, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-04-17T08:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 7, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 4566, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 2824, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 22, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-04-17T08:02:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 8, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-04-17T08:02:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-04-17T08:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 93, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-04-17T08:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 9, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 532, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 41 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-04-17T08:02:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 17, 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 10, 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 4030, 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 2022, 2024-04-17T08:02:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-04-17T08:02:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-17T08:02:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 33235, 2024-04-17T08:02:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 7902058, 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 33235, 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 33235, 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 7902058, 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-04-17T08:02:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-04-17T08:02:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 8483, 2024-04-17T08:02:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 37826804, 2024-04-17T08:02:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 8483, 2024-04-17T08:02:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 37826804, 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-04-17T08:02:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 8483, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 8483, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 8483, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 8483, 2024-04-17T08:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-04-17T08:02:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 2022, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 220286, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 2022, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 48, 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 220286, 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x561a29e80460": { 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 16138, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 34356594, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 16138, 2024-04-17T08:02:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 641, 2024-04-17T08:02:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 34356594, 2024-04-17T08:02:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:36.562 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-17T08:02:37.234 INFO:tasks.check_counter:Daemon mds.b mds.exported=144 2024-04-17T08:02:37.234 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-17T08:02:37.234 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=23 2024-04-17T08:02:37.234 DEBUG:tasks.check_counter:Getting stats from e 2024-04-17T08:02:37.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-17T08:02:37.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:37 smithi053 ceph-mon[36437]: pgmap v540: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-17T08:02:37.491 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T08:02:37.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:37 smithi062 ceph-mon[36507]: pgmap v540: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-17T08:02:37.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:37 smithi046 ceph-mon[32038]: pgmap v540: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-17T08:02:39.318 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:39 smithi053 ceph-mon[36437]: pgmap v541: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-17T08:02:39.357 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-17T08:02:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 0, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 0, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 0, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001249745, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T08:02:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000112714, 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 5, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 98, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3152, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 95312, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 3, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.006136539, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.004126689, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000891173, 2024-04-17T08:02:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 97, 2024-04-17T08:02:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.007007207, 2024-04-17T08:02:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000072239 2024-04-17T08:02:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 95312 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 128, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 125, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 454400, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 76688, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.022214166, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.007986177, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.011659454, 2024-04-17T08:02:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.003269632, 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 124, 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.007604733, 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000061328 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 454400, 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 76688 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 148801, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 27, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 129, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 436330, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 129, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 129, 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 436330, 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:39.376 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-17T08:02:39.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:39 smithi062 ceph-mon[36507]: pgmap v541: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-17T08:02:39.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:39 smithi046 ceph-mon[32038]: pgmap v541: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-17T08:02:40.068 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-17T08:02:40.069 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-17T08:02:40.069 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-17T08:02:40.069 DEBUG:tasks.check_counter:Getting stats from h 2024-04-17T08:02:40.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-17T08:02:40.322 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T08:02:41.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:41 smithi053 ceph-mon[36437]: pgmap v542: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-17T08:02:41.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:41 smithi062 ceph-mon[36507]: pgmap v542: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-17T08:02:41.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:41 smithi046 ceph-mon[32038]: pgmap v542: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-17T08:02:42.042 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-17T08:02:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 4270, 2024-04-17T08:02:42.042 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 6358, 2024-04-17T08:02:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3763099, 2024-04-17T08:02:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 6138019, 2024-04-17T08:02:42.043 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 14, 2024-04-17T08:02:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-04-17T08:02:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.382255284, 2024-04-17T08:02:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.169354005, 2024-04-17T08:02:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.176266836, 2024-04-17T08:02:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.041822562, 2024-04-17T08:02:42.044 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6362, 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.277795614, 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000043664 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6861, 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001419282, 2024-04-17T08:02:42.045 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000206 2024-04-17T08:02:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 502480, 2024-04-17T08:02:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 85168 2024-04-17T08:02:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 5436, 2024-04-17T08:02:42.046 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 9312, 2024-04-17T08:02:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1706799, 2024-04-17T08:02:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 8184845, 2024-04-17T08:02:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 18, 2024-04-17T08:02:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-04-17T08:02:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.354003965, 2024-04-17T08:02:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.159125380, 2024-04-17T08:02:42.047 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.185643546, 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.035271166, 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 9310, 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.283650841, 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000030467 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4820, 2024-04-17T08:02:42.048 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001831154, 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000379 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 315248 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1579, 2024-04-17T08:02:42.049 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1708, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 541721, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 1325767, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 14, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 7, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.105667000, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.035588833, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.035467553, 2024-04-17T08:02:42.050 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.028361525, 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1702, 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.073242422, 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000043033 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 544, 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000106878, 2024-04-17T08:02:42.051 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000196 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 736 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-17T08:02:42.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2254, 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.553601105, 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000689264 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.053 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1109, 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.092471906, 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000083383 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-04-17T08:02:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 2320, 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 2312, 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2312, 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 10.440601193, 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004515830 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-04-17T08:02:42.055 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 8, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 11, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 1401, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 54, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 24, 2024-04-17T08:02:42.056 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 1323, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 1323, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 1025, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 377, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 377, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 56, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 4654, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 2989, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 8, 2024-04-17T08:02:42.057 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 53, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 87, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 34, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 945, 2024-04-17T08:02:42.058 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 87, 2024-04-17T08:02:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 2198, 2024-04-17T08:02:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-04-17T08:02:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 2643, 2024-04-17T08:02:42.059 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-04-17T08:02:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 3452, 2024-04-17T08:02:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 26581383, 2024-04-17T08:02:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-04-17T08:02:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 2643, 2024-04-17T08:02:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-04-17T08:02:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T08:02:42.060 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 42, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 42, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 2601, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 39, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 1143, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T08:02:42.061 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 1646, 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 1390, 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 85, 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 1286 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-04-17T08:02:42.062 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 54, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 69, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 614, 2024-04-17T08:02:42.063 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 614, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-04-17T08:02:42.064 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 34, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.065 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-04-17T08:02:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-04-17T08:02:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 4373, 2024-04-17T08:02:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-04-17T08:02:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-04-17T08:02:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 4373, 2024-04-17T08:02:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-04-17T08:02:42.066 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-04-17T08:02:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 7, 2024-04-17T08:02:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-04-17T08:02:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-04-17T08:02:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 7, 2024-04-17T08:02:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-04-17T08:02:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-04-17T08:02:42.067 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 11701202, 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 178, 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.316586050, 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.007396550 2024-04-17T08:02:42.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-04-17T08:02:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-04-17T08:02:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1084, 2024-04-17T08:02:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 2365, 2024-04-17T08:02:42.069 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 1281, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 140, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 144, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 4, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 1401, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 3047, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 1646, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 377, 2024-04-17T08:02:42.070 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 2464, 2024-04-17T08:02:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 2087, 2024-04-17T08:02:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 67652, 2024-04-17T08:02:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 223608 2024-04-17T08:02:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-04-17T08:02:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 3449, 2024-04-17T08:02:42.071 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 2320, 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 8, 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 1088, 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 424, 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.163819111, 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002744856 2024-04-17T08:02:42.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-04-17T08:02:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.073 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-04-17T08:02:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-04-17T08:02:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.074 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-04-17T08:02:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1147, 2024-04-17T08:02:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.408164074, 2024-04-17T08:02:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000355853 2024-04-17T08:02:42.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-04-17T08:02:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-04-17T08:02:42.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-04-17T08:02:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.077 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-04-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.079 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006115690, 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003057845 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-04-17T08:02:42.080 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 94, 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.030212332, 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000321407 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 29, 2024-04-17T08:02:42.082 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.032104324, 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001107045 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 7.643088561, 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 3.821544280 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.083 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-04-17T08:02:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.087 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.089 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-04-17T08:02:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 614, 2024-04-17T08:02:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.157097101, 2024-04-17T08:02:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001884522 2024-04-17T08:02:42.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 2, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 1, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 1499, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 1499, 2024-04-17T08:02:42.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 1968, 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:42.093 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:42.094 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:42.095 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:42.096 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 331937, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 1191, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 704, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 8, 2024-04-17T08:02:42.097 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8888, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 170, 2024-04-17T08:02:42.098 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 3935500, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 46055, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-17T08:02:42.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 4183, 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 7741209, 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 28, 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 4183, 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-04-17T08:02:42.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4183, 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 16.582655226, 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003964297 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4183, 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4253 2024-04-17T08:02:42.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 4183, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 2659, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 1524, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-04-17T08:02:42.102 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 16, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 194, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 21, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 1241, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-04-17T08:02:42.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 2643, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-04-17T08:02:42.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-04-17T08:02:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-04-17T08:02:42.106 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 93, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 4, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-04-17T08:02:42.107 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 87, 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 21 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-04-17T08:02:42.108 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-04-17T08:02:42.109 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 16, 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 8, 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 1228, 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 614, 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:42.110 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 11281, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 5049385, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 11281, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:42.111 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 11281, 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 5049385, 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-04-17T08:02:42.112 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:42.113 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-17T08:02:42.113 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:42.113 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-04-17T08:02:42.113 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-04-17T08:02:42.114 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:42.114 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:42.114 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 4183, 2024-04-17T08:02:42.114 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 7808135, 2024-04-17T08:02:42.114 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4183, 2024-04-17T08:02:42.114 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 7808135, 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:42.115 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 4183, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 4183, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 4183, 2024-04-17T08:02:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 4183, 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:42.117 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 614, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 66926, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 614, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:42.118 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 16, 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 66926, 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55d2f7e08460": { 2024-04-17T08:02:42.119 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 4373, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 7506898, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 4373, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 178, 2024-04-17T08:02:42.120 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 7506898, 2024-04-17T08:02:42.121 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:42.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:42.121 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:42.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:42.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:42.121 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-17T08:02:42.855 INFO:tasks.check_counter:Daemon mds.h mds.exported=34 2024-04-17T08:02:42.855 INFO:tasks.check_counter:Daemon mds.h mds.imported=87 2024-04-17T08:02:42.855 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-17T08:02:42.855 DEBUG:tasks.check_counter:Getting stats from k 2024-04-17T08:02:42.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-17T08:02:43.108 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T08:02:43.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:43 smithi053 ceph-mon[36437]: pgmap v543: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-17T08:02:43.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:43 smithi062 ceph-mon[36507]: pgmap v543: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-17T08:02:43.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:43 smithi046 ceph-mon[32038]: pgmap v543: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-17T08:02:44.991 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-04-17T08:02:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 4, 2024-04-17T08:02:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 4, 2024-04-17T08:02:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2992, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 448, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001419735, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000035901, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000524997, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000678982, 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000226327 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1, 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 99, 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 160, 2024-04-17T08:02:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 95392, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 3, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.006260118, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.004254941, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000640380, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 97, 2024-04-17T08:02:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.005992020, 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000061773 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 95392 2024-04-17T08:02:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 132, 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 129, 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 455488, 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 79248, 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-04-17T08:02:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.022543612, 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.007584507, 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.012533900, 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.003385462, 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 128, 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006971733, 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000054466 2024-04-17T08:02:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 455488, 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 79248 2024-04-17T08:02:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 148823, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 27, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-04-17T08:02:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 133, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 436850, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 133, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-04-17T08:02:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 133, 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 436850, 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-04-17T08:02:45.009 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-04-17T08:02:45.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:45 smithi053 ceph-mon[36437]: pgmap v544: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T08:02:45.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:45 smithi062 ceph-mon[36507]: pgmap v544: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T08:02:45.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:45 smithi046 ceph-mon[32038]: pgmap v544: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-17T08:02:45.651 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-17T08:02:45.651 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-17T08:02:45.651 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-17T08:02:45.652 DEBUG:tasks.check_counter:Getting stats from c 2024-04-17T08:02:45.652 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-17T08:02:45.915 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T08:02:47.220 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:47 smithi053 ceph-mon[36437]: pgmap v545: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T08:02:47.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:47 smithi062 ceph-mon[36507]: pgmap v545: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T08:02:47.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:47 smithi046 ceph-mon[32038]: pgmap v545: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-17T08:02:48.091 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-17T08:02:48.092 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:48.092 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 557, 2024-04-17T08:02:48.092 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 549, 2024-04-17T08:02:48.092 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 631456, 2024-04-17T08:02:48.092 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 210358, 2024-04-17T08:02:48.092 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.077537952, 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.023686452, 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.027836474, 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.021223291, 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 548, 2024-04-17T08:02:48.093 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.029093273, 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000053089 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.094 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 492688, 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 79024 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 7, 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 101, 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3708, 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 316298, 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-17T08:02:48.095 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.007394815, 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.004454590, 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001743115, 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000076343, 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 100, 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.009473496, 2024-04-17T08:02:48.096 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000094734 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 315600 2024-04-17T08:02:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 775, 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 777, 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 7463549, 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 246457, 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-17T08:02:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.064167094, 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.019978322, 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.019058508, 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.020194102, 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 775, 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.030824116, 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000039773 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.099 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.100 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2044, 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.060041568, 2024-04-17T08:02:48.101 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000029374 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.102 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.103 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-17T08:02:48.104 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 4118, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 181, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 3937, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 3860, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 39, 2024-04-17T08:02:48.105 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-17T08:02:48.106 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-17T08:02:48.107 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T08:02:48.108 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-04-17T08:02:48.109 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-17T08:02:48.110 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-17T08:02:48.111 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-17T08:02:48.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-17T08:02:48.113 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 4042, 2024-04-17T08:02:48.114 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-17T08:02:48.114 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-17T08:02:48.114 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-04-17T08:02:48.114 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-17T08:02:48.114 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-17T08:02:48.114 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 6, 2024-04-17T08:02:48.114 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 11459918, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 11459857, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.115 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 4042 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2396, 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 3003, 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 607, 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 180, 2024-04-17T08:02:48.116 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 180, 2024-04-17T08:02:48.117 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-04-17T08:02:48.117 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 4118, 2024-04-17T08:02:48.117 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 4118, 2024-04-17T08:02:48.117 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 0, 2024-04-17T08:02:48.117 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-04-17T08:02:48.117 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-04-17T08:02:48.117 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 57636, 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215416 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-04-17T08:02:48.118 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.119 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-17T08:02:48.120 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.121 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-17T08:02:48.122 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.122 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.122 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.122 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.122 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-17T08:02:48.122 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.122 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.123 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.125 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-17T08:02:48.125 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.125 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.125 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.125 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.125 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-17T08:02:48.126 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.126 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.126 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.126 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.128 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-17T08:02:48.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.130 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-17T08:02:48.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.131 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.131 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-17T08:02:48.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.132 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.134 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.135 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.137 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 2, 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 1, 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-04-17T08:02:48.138 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 1941, 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:48.139 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:48.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:48.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:48.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 269404, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 2429, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:48.143 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 8888, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 170, 2024-04-17T08:02:48.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 8704486, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 74376, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:48.145 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 1190, 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-04-17T08:02:48.146 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 1190, 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1190, 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.614990646, 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000516798 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1190, 2024-04-17T08:02:48.147 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1191 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 1190, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 1190, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 739, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-04-17T08:02:48.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 449, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-04-17T08:02:48.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-17T08:02:48.150 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-17T08:02:48.151 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-17T08:02:48.152 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 93, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 4, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 4, 2024-04-17T08:02:48.153 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 4, 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-17T08:02:48.154 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-17T08:02:48.155 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:48.156 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 1335, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 7968213, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 1335, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:48.157 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1335, 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 7968213, 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-17T08:02:48.158 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1190, 2024-04-17T08:02:48.159 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 7265619, 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1190, 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 7265619, 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1190, 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 1190, 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1190, 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1190, 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:48.163 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.164 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x56173a5946e0": { 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:48.165 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:48.166 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:48.167 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:48.167 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.167 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:48.167 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-17T08:02:48.920 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-17T08:02:48.920 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-17T08:02:48.920 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-17T08:02:48.921 DEBUG:tasks.check_counter:Getting stats from f 2024-04-17T08:02:48.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-17T08:02:49.181 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T08:02:49.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:49 smithi062 ceph-mon[36507]: pgmap v546: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-17T08:02:49.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:49 smithi053 ceph-mon[36437]: pgmap v546: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-17T08:02:49.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:49 smithi046 ceph-mon[32038]: pgmap v546: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-17T08:02:51.047 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-17T08:02:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 330, 2024-04-17T08:02:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 331, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 4281319, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 104686, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.041386131, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.010450756, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.014356125, 2024-04-17T08:02:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.011950394, 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 329, 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.016747890, 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000050905 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 357, 2024-04-17T08:02:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 454, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3410135, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 351837, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.049813228, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.015481639, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.014549406, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.015032777, 2024-04-17T08:02:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 452, 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.019798170, 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000043801 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 238624 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 668, 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 658, 2024-04-17T08:02:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 660976, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 245154, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.088325938, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.027375035, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.030506028, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.025247080, 2024-04-17T08:02:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 658, 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.037518832, 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000057019 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 495952, 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 80112 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2029, 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.067699032, 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000033365 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-17T08:02:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-17T08:02:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-17T08:02:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 1930, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 1930, 2024-04-17T08:02:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 1930, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 55, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-04-17T08:02:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-17T08:02:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-17T08:02:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-17T08:02:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-04-17T08:02:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-17T08:02:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-17T08:02:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-17T08:02:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-17T08:02:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 4377, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 7, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-04-17T08:02:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 11709305, 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 11707277, 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 4377 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1076, 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 2356, 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 1280, 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 100, 2024-04-17T08:02:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 134, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 34, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 1930, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 2876, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 946, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 55080, 2024-04-17T08:02:51.073 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215416 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-17T08:02:51.074 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.076 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-17T08:02:51.077 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.078 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.079 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-17T08:02:51.080 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.081 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.081 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.081 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-17T08:02:51.081 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-17T08:02:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.082 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.083 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-17T08:02:51.084 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.086 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-17T08:02:51.087 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.088 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.090 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-17T08:02:51.091 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.092 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 2, 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 1, 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-17T08:02:51.093 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 1943, 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:51.094 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:51.095 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:51.096 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:51.097 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:51.098 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 236724, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 1117, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:51.099 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 8888, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 170, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 3994994, 2024-04-17T08:02:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 36334, 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-17T08:02:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 1204, 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 1204, 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1204, 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.419978413, 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001179384 2024-04-17T08:02:51.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1204, 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1205 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 1204, 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 1204, 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-04-17T08:02:51.103 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 743, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 459, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-17T08:02:51.104 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-04-17T08:02:51.105 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-17T08:02:51.106 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-17T08:02:51.107 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-17T08:02:51.108 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 93, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 4, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 6, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 6, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-04-17T08:02:51.109 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-04-17T08:02:51.111 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 1351, 2024-04-17T08:02:51.112 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 8220438, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 1351, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1351, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 8220438, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.113 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:51.114 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1204, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 7515030, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1204, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 7515030, 2024-04-17T08:02:51.115 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-17T08:02:51.116 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1204, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 1204, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1204, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1204, 2024-04-17T08:02:51.117 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:51.118 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-17T08:02:51.119 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x564b6c2de6e0": { 2024-04-17T08:02:51.120 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:51.121 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:51.122 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-17T08:02:51.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:51 smithi053 ceph-mon[36437]: pgmap v547: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-17T08:02:51.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:51 smithi062 ceph-mon[36507]: pgmap v547: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-17T08:02:51.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:51 smithi046 ceph-mon[32038]: pgmap v547: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-17T08:02:51.845 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-17T08:02:51.846 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-17T08:02:51.846 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-17T08:02:51.846 DEBUG:tasks.check_counter:Getting stats from i 2024-04-17T08:02:51.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-17T08:02:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T08:02:53.260 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:53 smithi062 ceph-mon[36507]: pgmap v548: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-17T08:02:53.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:53 smithi053 ceph-mon[36437]: pgmap v548: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-17T08:02:53.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:53 smithi046 ceph-mon[32038]: pgmap v548: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-17T08:02:53.853 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-17T08:02:53.853 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:53.853 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 910, 2024-04-17T08:02:53.853 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 910, 2024-04-17T08:02:53.853 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 17047444, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 288566, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.105488716, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.031394438, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.033912097, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.029827543, 2024-04-17T08:02:53.854 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 909, 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.039049120, 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000042958 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.855 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 293, 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 285, 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 9740258, 2024-04-17T08:02:53.856 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 126314, 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.043454846, 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.012628533, 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.022526599, 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.007515126, 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:53.857 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 284, 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.017287131, 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000060870 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 492848, 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 79184 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 323, 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 419, 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 8473407, 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 342461, 2024-04-17T08:02:53.859 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-17T08:02:53.860 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-17T08:02:53.860 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.044347679, 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.015644903, 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.013796384, 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.010827637, 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 417, 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.020336647, 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000048768 2024-04-17T08:02:53.861 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 240064 2024-04-17T08:02:53.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-17T08:02:53.863 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2270, 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.067362704, 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000029675 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-17T08:02:53.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-17T08:02:53.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-17T08:02:53.866 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 2193, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 2193, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 2193, 2024-04-17T08:02:53.867 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 22, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-17T08:02:53.868 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-17T08:02:53.869 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 3452, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 26581383, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-17T08:02:53.870 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T08:02:53.871 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.872 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-17T08:02:53.873 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-17T08:02:53.874 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-17T08:02:53.875 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-17T08:02:53.876 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 16141, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 21, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 3, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-17T08:02:53.877 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 38568842, 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 38566801, 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 16141 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 118, 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 5982, 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 5864, 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 52, 2024-04-17T08:02:53.879 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 278, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 226, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 2193, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 6725, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 4532, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-04-17T08:02:53.880 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 62968, 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215416 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-04-17T08:02:53.881 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.882 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-17T08:02:53.883 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.885 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-17T08:02:53.886 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.888 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-17T08:02:53.888 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.888 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.888 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.889 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-17T08:02:53.889 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.889 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.889 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.890 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.891 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.893 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.894 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.895 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.896 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.897 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.898 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-17T08:02:53.899 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.900 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 3, 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 2, 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-17T08:02:53.901 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 1947, 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:53.903 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:53.904 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:53.905 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:53.906 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 217833, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 159, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:53.907 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 8888, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 170, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-17T08:02:53.908 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 1887187, 2024-04-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 11884, 2024-04-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.909 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 1376, 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 26, 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 4, 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 1376, 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-17T08:02:53.910 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1376, 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.621975327, 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001178761 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1376, 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1377 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.911 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 1376, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 1376, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 862, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 511, 2024-04-17T08:02:53.912 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-17T08:02:53.913 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 2, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-17T08:02:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-17T08:02:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-17T08:02:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 93, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 4, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 8, 2024-04-17T08:02:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 8, 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-17T08:02:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 1522, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 35113733, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 1522, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1522, 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 35113733, 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1376, 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 34380123, 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1376, 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 34380123, 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 1376, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 1376, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 1376, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1376, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:53.927 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:53.928 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x55a6b998c6e0": { 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:53.929 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-17T08:02:53.930 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:53.931 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:53.931 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:53.931 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:53.931 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.931 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:53.931 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-17T08:02:54.674 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-17T08:02:54.674 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-17T08:02:54.674 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-17T08:02:54.674 DEBUG:tasks.check_counter:Getting stats from l 2024-04-17T08:02:54.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-17T08:02:54.925 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T08:02:55.302 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:55 smithi062 ceph-mon[36507]: pgmap v549: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-17T08:02:55.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:55 smithi053 ceph-mon[36437]: pgmap v549: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-17T08:02:55.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:55 smithi046 ceph-mon[32038]: pgmap v549: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-17T08:02:56.760 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-17T08:02:56.760 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T08:02:56.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-04-17T08:02:56.760 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.001292481, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000259000, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:56.761 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:56.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2, 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 104, 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 320, 2024-04-17T08:02:56.763 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 174416, 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.007443756, 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.005016834, 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001037333, 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:56.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 101, 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.007909488, 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000078311 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.765 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 174416 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 141, 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 136, 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 459904, 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 82416, 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-04-17T08:02:56.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.023358795, 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.008101318, 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.012648274, 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.003387886, 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 136, 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.008155823, 2024-04-17T08:02:56.767 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000059969 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 459904, 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 82416 2024-04-17T08:02:56.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T08:02:56.769 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-17T08:02:56.770 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T08:02:56.771 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T08:02:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T08:02:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T08:02:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-17T08:02:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-17T08:02:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-17T08:02:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T08:02:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T08:02:56.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148841, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 28, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-17T08:02:56.774 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-17T08:02:56.775 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-17T08:02:56.775 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-17T08:02:56.775 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-04-17T08:02:56.775 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-04-17T08:02:56.775 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T08:02:56.776 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-17T08:02:56.776 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-17T08:02:56.776 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-17T08:02:56.776 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-04-17T08:02:56.777 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-04-17T08:02:56.777 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-17T08:02:56.777 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-17T08:02:56.777 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-17T08:02:56.778 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-17T08:02:56.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-17T08:02:56.778 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T08:02:56.778 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-17T08:02:56.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-17T08:02:56.779 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-17T08:02:56.779 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 139, 2024-04-17T08:02:56.779 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 437591, 2024-04-17T08:02:56.779 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-17T08:02:56.780 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 139, 2024-04-17T08:02:56.780 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-17T08:02:56.780 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-17T08:02:56.780 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 139, 2024-04-17T08:02:56.780 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 437591, 2024-04-17T08:02:56.781 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-17T08:02:56.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-17T08:02:56.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-17T08:02:56.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-17T08:02:56.782 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:56.782 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-17T08:02:56.782 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-17T08:02:57.440 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:57 smithi062 ceph-mon[36507]: pgmap v550: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-17T08:02:57.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:57 smithi046 ceph-mon[32038]: pgmap v550: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-17T08:02:57.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:57 smithi053 ceph-mon[36437]: pgmap v550: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-17T08:02:57.472 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-17T08:02:57.472 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-17T08:02:57.472 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-17T08:02:57.472 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-17T08:02:57.484 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-17T08:02:57.484 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T08:02:57.484 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T08:02:57.541 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-17T08:02:57.541 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T08:02:57.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T08:02:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 -- 172.21.15.46:0/2497290812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 msgr2=0x7fa0741034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 --2- 172.21.15.46:0/2497290812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 0x7fa0741034c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa060009900 tx=0x7fa06002bc30 comp rx=0 tx=0).stop 2024-04-17T08:02:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 -- 172.21.15.46:0/2497290812 shutdown_connections 2024-04-17T08:02:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 --2- 172.21.15.46:0/2497290812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa074103a00 0x7fa07410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 --2- 172.21.15.46:0/2497290812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 0x7fa0741034c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 --2- 172.21.15.46:0/2497290812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 0x7fa074102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 -- 172.21.15.46:0/2497290812 >> 172.21.15.46:0/2497290812 conn(0x7fa0740fd420 msgr2=0x7fa0740ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 -- 172.21.15.46:0/2497290812 shutdown_connections 2024-04-17T08:02:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.809+0000 7fa073e00640 1 -- 172.21.15.46:0/2497290812 wait complete. 2024-04-17T08:02:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.810+0000 7fa073e00640 1 Processor -- start 2024-04-17T08:02:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.810+0000 7fa073e00640 1 -- start start 2024-04-17T08:02:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.810+0000 7fa073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 0x7fa07419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.810+0000 7fa073e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 0x7fa07419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.810+0000 7fa073e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa074103a00 0x7fa0741a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa073e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa07410db50 con 0x7fa074101e60 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa073e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa07410d9d0 con 0x7fa074103040 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa073e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa07410dcd0 con 0x7fa074103a00 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 0x7fa07419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 0x7fa07419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35300/0 (socket says 172.21.15.46:35300) 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 -- 172.21.15.46:0/2752808443 learned_addr learned my addr 172.21.15.46:0/2752808443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa073400640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa074103a00 0x7fa0741a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072000640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 0x7fa07419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 -- 172.21.15.46:0/2752808443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa074103a00 msgr2=0x7fa0741a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa074103a00 0x7fa0741a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 -- 172.21.15.46:0/2752808443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 msgr2=0x7fa07419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 0x7fa07419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.811+0000 7fa072a00640 1 -- 172.21.15.46:0/2752808443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0741a1f70 con 0x7fa074101e60 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.812+0000 7fa072a00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 0x7fa07419a350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e9c0 tx=0x7fa06400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.812+0000 7fa06b400640 1 -- 172.21.15.46:0/2752808443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400cd80 con 0x7fa074101e60 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.812+0000 7fa06b400640 1 -- 172.21.15.46:0/2752808443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0640035f0 con 0x7fa074101e60 2024-04-17T08:02:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.812+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0741a2260 con 0x7fa074101e60 2024-04-17T08:02:57.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.812+0000 7fa06b400640 1 -- 172.21.15.46:0/2752808443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064010690 con 0x7fa074101e60 2024-04-17T08:02:57.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.812+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0741a27d0 con 0x7fa074101e60 2024-04-17T08:02:57.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.813+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0741a2a90 con 0x7fa074101e60 2024-04-17T08:02:57.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.816+0000 7fa06b400640 1 -- 172.21.15.46:0/2752808443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa0640037a0 con 0x7fa074101e60 2024-04-17T08:02:57.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.817+0000 7fa06b400640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa0580a3c20 0x7fa0580a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:57.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.818+0000 7fa06b400640 1 -- 172.21.15.46:0/2752808443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa0640d0af0 con 0x7fa074101e60 2024-04-17T08:02:57.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.818+0000 7fa072000640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa0580a3c20 0x7fa0580a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:57.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.819+0000 7fa072000640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa0580a3c20 0x7fa0580a60e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa06002c7b0 tx=0x7fa060005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:57.822+0000 7fa06b400640 1 -- 172.21.15.46:0/2752808443 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa06409d140 con 0x7fa074101e60 2024-04-17T08:02:58.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.032+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0741a2c60 con 0x7fa074101e60 2024-04-17T08:02:58.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.034+0000 7fa06b400640 1 -- 172.21.15.46:0/2752808443 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7fa06409ca40 con 0x7fa074101e60 2024-04-17T08:02:58.034 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:02:58.034 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T08:02:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa0580a3c20 msgr2=0x7fa0580a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa0580a3c20 0x7fa0580a60e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa06002c7b0 tx=0x7fa060005fd0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 msgr2=0x7fa07419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 0x7fa07419a350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa06400e9c0 tx=0x7fa06400ee80 comp rx=0 tx=0).stop 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 shutdown_connections 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fa0580a3c20 0x7fa0580a60e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa074103a00 0x7fa0741a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074103040 0x7fa07419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 --2- 172.21.15.46:0/2752808443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa074101e60 0x7fa07419a350 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 >> 172.21.15.46:0/2752808443 conn(0x7fa0740fd420 msgr2=0x7fa074074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 shutdown_connections 2024-04-17T08:02:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.037+0000 7fa073e00640 1 -- 172.21.15.46:0/2752808443 wait complete. 2024-04-17T08:02:58.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:50:07.527032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:55:24.035234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:55:24.942051+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:55:28.582645+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T08:02:58.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T08:02:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.281+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3371236307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 msgr2=0x7fb5f8077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.281+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3371236307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 0x7fb5f8077fc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009860 tx=0x7fb5e402bb70 comp rx=0 tx=0).stop 2024-04-17T08:02:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.282+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3371236307 shutdown_connections 2024-04-17T08:02:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.282+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3371236307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8078500 0x7fb5f8078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.282+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3371236307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 0x7fb5f8077fc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.282+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3371236307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5f8076920 0x7fb5f8076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.282+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3371236307 >> 172.21.15.46:0/3371236307 conn(0x7fb5f80ff720 msgr2=0x7fb5f8101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.282+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3371236307 shutdown_connections 2024-04-17T08:02:58.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.282+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3371236307 wait complete. 2024-04-17T08:02:58.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 Processor -- start 2024-04-17T08:02:58.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 -- start start 2024-04-17T08:02:58.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8076920 0x7fb5f819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 0x7fb5f819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5f8078500 0x7fb5f81a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8114200 con 0x7fb5f8078500 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8114080 con 0x7fb5f8077b60 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.283+0000 7fb5fcc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f8114380 con 0x7fb5f8076920 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8076920 0x7fb5f819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8076920 0x7fb5f819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50290/0 (socket says 172.21.15.46:50290) 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 -- 172.21.15.46:0/3987112204 learned_addr learned my addr 172.21.15.46:0/3987112204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f6800640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5f8078500 0x7fb5f81a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5400640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 0x7fb5f819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 -- 172.21.15.46:0/3987112204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 msgr2=0x7fb5f819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 0x7fb5f819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 -- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5f8078500 msgr2=0x7fb5f81a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5f8078500 0x7fb5f81a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 -- 172.21.15.46:0/3987112204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f81a6490 con 0x7fb5f8076920 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5400640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 0x7fb5f819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.284+0000 7fb5f5e00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8076920 0x7fb5f819e870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9c0 tx=0x7fb5e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.285+0000 7fb5eea00640 1 -- 172.21.15.46:0/3987112204 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800ccc0 con 0x7fb5f8076920 2024-04-17T08:02:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.285+0000 7fb5eea00640 1 -- 172.21.15.46:0/3987112204 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e8005500 con 0x7fb5f8076920 2024-04-17T08:02:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.285+0000 7fb5eea00640 1 -- 172.21.15.46:0/3987112204 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8010430 con 0x7fb5f8076920 2024-04-17T08:02:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.285+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f81a6780 con 0x7fb5f8076920 2024-04-17T08:02:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.285+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f81a6c50 con 0x7fb5f8076920 2024-04-17T08:02:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.287+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f8110b20 con 0x7fb5f8076920 2024-04-17T08:02:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.288+0000 7fb5eea00640 1 -- 172.21.15.46:0/3987112204 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb5e80046c0 con 0x7fb5f8076920 2024-04-17T08:02:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.289+0000 7fb5eea00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb5dc0a3bb0 0x7fb5dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.289+0000 7fb5eea00640 1 -- 172.21.15.46:0/3987112204 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb5e80d0a90 con 0x7fb5f8076920 2024-04-17T08:02:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.290+0000 7fb5f5400640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb5dc0a3bb0 0x7fb5dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.291+0000 7fb5f5400640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb5dc0a3bb0 0x7fb5dc0a6070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009830 tx=0x7fb5e402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:58.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.294+0000 7fb5eea00640 1 -- 172.21.15.46:0/3987112204 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb5e809d080 con 0x7fb5f8076920 2024-04-17T08:02:58.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:58 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2752808443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:02:58.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.504+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5f8062b60 con 0x7fb5f8076920 2024-04-17T08:02:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.505+0000 7fb5eea00640 1 -- 172.21.15.46:0/3987112204 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb5e809c980 con 0x7fb5f8076920 2024-04-17T08:02:58.504 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-17T08:02:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb5dc0a3bb0 msgr2=0x7fb5dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb5dc0a3bb0 0x7fb5dc0a6070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009830 tx=0x7fb5e402e040 comp rx=0 tx=0).stop 2024-04-17T08:02:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8076920 msgr2=0x7fb5f819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8076920 0x7fb5f819e870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9c0 tx=0x7fb5e800ee80 comp rx=0 tx=0).stop 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 shutdown_connections 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7fb5dc0a3bb0 0x7fb5dc0a6070 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5f8078500 0x7fb5f81a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5f8077b60 0x7fb5f819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 --2- 172.21.15.46:0/3987112204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5f8076920 0x7fb5f819e870 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 >> 172.21.15.46:0/3987112204 conn(0x7fb5f80ff720 msgr2=0x7fb5f8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 shutdown_connections 2024-04-17T08:02:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.508+0000 7fb5fcc00640 1 -- 172.21.15.46:0/3987112204 wait complete. 2024-04-17T08:02:58.523 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:58 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2752808443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:02:58.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T08:02:58.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:58 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2752808443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:02:58.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.737+0000 7f670a400640 1 -- 172.21.15.46:0/1219751480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704101de0 msgr2=0x7f6704102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.737+0000 7f670a400640 1 --2- 172.21.15.46:0/1219751480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704101de0 0x7f6704102200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009900 tx=0x7f66f402bc50 comp rx=0 tx=0).stop 2024-04-17T08:02:58.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.737+0000 7f670a400640 1 -- 172.21.15.46:0/1219751480 shutdown_connections 2024-04-17T08:02:58.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.737+0000 7f670a400640 1 --2- 172.21.15.46:0/1219751480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6704103990 0x7f670410a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.737+0000 7f670a400640 1 --2- 172.21.15.46:0/1219751480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6704102fd0 0x7f6704103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.737+0000 7f670a400640 1 --2- 172.21.15.46:0/1219751480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704101de0 0x7f6704102200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.737+0000 7f670a400640 1 -- 172.21.15.46:0/1219751480 >> 172.21.15.46:0/1219751480 conn(0x7f67040fd370 msgr2=0x7f67040ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.738+0000 7f670a400640 1 -- 172.21.15.46:0/1219751480 shutdown_connections 2024-04-17T08:02:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.738+0000 7f670a400640 1 -- 172.21.15.46:0/1219751480 wait complete. 2024-04-17T08:02:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 Processor -- start 2024-04-17T08:02:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 -- start start 2024-04-17T08:02:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6704101de0 0x7f670419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704102fd0 0x7f670419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6704103990 0x7f67041a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f670410da30 con 0x7f6704102fd0 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f670410d8b0 con 0x7f6704101de0 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f670a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f670410dbb0 con 0x7f6704103990 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704102fd0 0x7f670419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704102fd0 0x7f670419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39434/0 (socket says 172.21.15.46:39434) 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f66fbe00640 1 -- 172.21.15.46:0/3766753703 learned_addr learned my addr 172.21.15.46:0/3766753703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f6709000640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6704101de0 0x7f670419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.739+0000 7f6709a00640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6704103990 0x7f67041a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f66fbe00640 1 -- 172.21.15.46:0/3766753703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6704103990 msgr2=0x7f67041a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f66fbe00640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6704103990 0x7f67041a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f66fbe00640 1 -- 172.21.15.46:0/3766753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6704101de0 msgr2=0x7f670419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f66fbe00640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6704101de0 0x7f670419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f66fbe00640 1 -- 172.21.15.46:0/3766753703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67041a2000 con 0x7f6704102fd0 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f66fbe00640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704102fd0 0x7f670419a920 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e9f0 tx=0x7f66ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f6702000640 1 -- 172.21.15.46:0/3766753703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00ccc0 con 0x7f6704102fd0 2024-04-17T08:02:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f6702000640 1 -- 172.21.15.46:0/3766753703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec010910 con 0x7f6704102fd0 2024-04-17T08:02:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67041a22f0 con 0x7f6704102fd0 2024-04-17T08:02:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.740+0000 7f6702000640 1 -- 172.21.15.46:0/3766753703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0188a0 con 0x7f6704102fd0 2024-04-17T08:02:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.741+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a2840 con 0x7f6704102fd0 2024-04-17T08:02:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.742+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67041a2aa0 con 0x7f6704102fd0 2024-04-17T08:02:58.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.744+0000 7f6702000640 1 -- 172.21.15.46:0/3766753703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f66ec010ac0 con 0x7f6704102fd0 2024-04-17T08:02:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.746+0000 7f6702000640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66e80a3bb0 0x7f66e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.746+0000 7f6702000640 1 -- 172.21.15.46:0/3766753703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f66ec0d0c20 con 0x7f6704102fd0 2024-04-17T08:02:58.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.747+0000 7f6709000640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66e80a3bb0 0x7f66e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:58.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.747+0000 7f6709000640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66e80a3bb0 0x7f66e80a6070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f66f4000c00 tx=0x7f66f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.751+0000 7f6702000640 1 -- 172.21.15.46:0/3766753703 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f66ec09d210 con 0x7f6704102fd0 2024-04-17T08:02:58.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.957+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67041a2c70 con 0x7f6704102fd0 2024-04-17T08:02:58.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.957+0000 7f6702000640 1 -- 172.21.15.46:0/3766753703 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f66ec09cb10 con 0x7f6704102fd0 2024-04-17T08:02:58.956 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T08:02:58.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66e80a3bb0 msgr2=0x7f66e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66e80a3bb0 0x7f66e80a6070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f66f4000c00 tx=0x7f66f4005fd0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704102fd0 msgr2=0x7f670419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:58.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704102fd0 0x7f670419a920 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e9f0 tx=0x7f66ec00edf0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 shutdown_connections 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66e80a3bb0 0x7f66e80a6070 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6704103990 0x7f67041a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6704102fd0 0x7f670419a920 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 --2- 172.21.15.46:0/3766753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6704101de0 0x7f670419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 >> 172.21.15.46:0/3766753703 conn(0x7f67040fd370 msgr2=0x7f67041087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 shutdown_connections 2024-04-17T08:02:58.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:58.960+0000 7f670a400640 1 -- 172.21.15.46:0/3766753703 wait complete. 2024-04-17T08:02:58.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T08:02:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 -- 172.21.15.46:0/3794779693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481053c0 msgr2=0x7f6648105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 --2- 172.21.15.46:0/3794779693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481053c0 0x7f6648105840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6630009860 tx=0x7f663002bb70 comp rx=0 tx=0).stop 2024-04-17T08:02:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 -- 172.21.15.46:0/3794779693 shutdown_connections 2024-04-17T08:02:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 --2- 172.21.15.46:0/3794779693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6648105d80 0x7f664810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 --2- 172.21.15.46:0/3794779693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481053c0 0x7f6648105840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 --2- 172.21.15.46:0/3794779693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481041d0 0x7f66481045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 -- 172.21.15.46:0/3794779693 >> 172.21.15.46:0/3794779693 conn(0x7f66480ff780 msgr2=0x7f6648101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 -- 172.21.15.46:0/3794779693 shutdown_connections 2024-04-17T08:02:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.174+0000 7f664d000640 1 -- 172.21.15.46:0/3794779693 wait complete. 2024-04-17T08:02:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 Processor -- start 2024-04-17T08:02:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 -- start start 2024-04-17T08:02:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481041d0 0x7f664819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481053c0 0x7f664819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6648105d80 0x7f66481a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f664810fe60 con 0x7f66481053c0 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f664810fce0 con 0x7f6648105d80 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f664d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f664810ffe0 con 0x7f66481041d0 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f663d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481053c0 0x7f664819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f6645e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481041d0 0x7f664819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f663d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481053c0 0x7f664819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39458/0 (socket says 172.21.15.46:39458) 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.176+0000 7f663d400640 1 -- 172.21.15.46:0/2313320135 learned_addr learned my addr 172.21.15.46:0/2313320135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6646800640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6648105d80 0x7f66481a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6645e00640 1 -- 172.21.15.46:0/2313320135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6648105d80 msgr2=0x7f66481a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6645e00640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6648105d80 0x7f66481a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6645e00640 1 -- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481053c0 msgr2=0x7f664819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6645e00640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481053c0 0x7f664819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6645e00640 1 -- 172.21.15.46:0/2313320135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66481a2080 con 0x7f66481041d0 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6646800640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6648105d80 0x7f66481a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f663d400640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481053c0 0x7f664819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f6645e00640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481041d0 0x7f664819a460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6648105220 tx=0x7f663800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.177+0000 7f663f400640 1 -- 172.21.15.46:0/2313320135 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638004440 con 0x7f66481041d0 2024-04-17T08:02:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.178+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66481a2370 con 0x7f66481041d0 2024-04-17T08:02:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.178+0000 7f663f400640 1 -- 172.21.15.46:0/2313320135 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663800cda0 con 0x7f66481041d0 2024-04-17T08:02:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.178+0000 7f663f400640 1 -- 172.21.15.46:0/2313320135 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66380196e0 con 0x7f66481041d0 2024-04-17T08:02:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.178+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66481a2910 con 0x7f66481041d0 2024-04-17T08:02:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.179+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664810b490 con 0x7f66481041d0 2024-04-17T08:02:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.181+0000 7f663f400640 1 -- 172.21.15.46:0/2313320135 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6638003780 con 0x7f66481041d0 2024-04-17T08:02:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.182+0000 7f663f400640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f662c0a3bb0 0x7f662c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.182+0000 7f663f400640 1 -- 172.21.15.46:0/2313320135 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f66380d0bd0 con 0x7f66481041d0 2024-04-17T08:02:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.183+0000 7f663d400640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f662c0a3bb0 0x7f662c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.183+0000 7f663d400640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f662c0a3bb0 0x7f662c0a6070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f664819b970 tx=0x7f663002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:59.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.187+0000 7f663f400640 1 -- 172.21.15.46:0/2313320135 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6638003e20 con 0x7f66481041d0 2024-04-17T08:02:59.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:59 smithi046 ceph-mon[32038]: pgmap v551: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-04-17T08:02:59.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:59 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3987112204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T08:02:59.359 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:02:59 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3766753703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T08:02:59.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.387+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6648062b60 con 0x7f66481041d0 2024-04-17T08:02:59.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.388+0000 7f663f400640 1 -- 172.21.15.46:0/2313320135 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f6638009510 con 0x7f66481041d0 2024-04-17T08:02:59.387 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-17T08:02:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f662c0a3bb0 msgr2=0x7f662c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f662c0a3bb0 0x7f662c0a6070 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f664819b970 tx=0x7f663002e040 comp rx=0 tx=0).stop 2024-04-17T08:02:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481041d0 msgr2=0x7f664819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481041d0 0x7f664819a460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6648105220 tx=0x7f663800ee50 comp rx=0 tx=0).stop 2024-04-17T08:02:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 shutdown_connections 2024-04-17T08:02:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f662c0a3bb0 0x7f662c0a6070 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6648105d80 0x7f66481a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66481053c0 0x7f664819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 --2- 172.21.15.46:0/2313320135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66481041d0 0x7f664819a460 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 >> 172.21.15.46:0/2313320135 conn(0x7f66480ff780 msgr2=0x7f6648108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 shutdown_connections 2024-04-17T08:02:59.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.391+0000 7f664d000640 1 -- 172.21.15.46:0/2313320135 wait complete. 2024-04-17T08:02:59.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T08:02:59.454 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:59 smithi062 ceph-mon[36507]: pgmap v551: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-04-17T08:02:59.455 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:59 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3987112204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T08:02:59.455 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:02:59 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3766753703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T08:02:59.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:59 smithi053 ceph-mon[36437]: pgmap v551: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-04-17T08:02:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:59 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3987112204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T08:02:59.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:02:59 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3766753703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T08:02:59.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.640+0000 7f6ed3e00640 1 -- 172.21.15.46:0/2066622193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4103fe0 msgr2=0x7f6ed4104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.640+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/2066622193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4103fe0 0x7f6ed4104400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009900 tx=0x7f6ec402bc30 comp rx=0 tx=0).stop 2024-04-17T08:02:59.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.640+0000 7f6ed3e00640 1 -- 172.21.15.46:0/2066622193 shutdown_connections 2024-04-17T08:02:59.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.640+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/2066622193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed4105b90 0x7f6ed410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.640+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/2066622193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed41051d0 0x7f6ed4105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.640+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/2066622193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4103fe0 0x7f6ed4104400 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.640+0000 7f6ed3e00640 1 -- 172.21.15.46:0/2066622193 >> 172.21.15.46:0/2066622193 conn(0x7f6ed40ff570 msgr2=0x7f6ed4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:59.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.641+0000 7f6ed3e00640 1 -- 172.21.15.46:0/2066622193 shutdown_connections 2024-04-17T08:02:59.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.641+0000 7f6ed3e00640 1 -- 172.21.15.46:0/2066622193 wait complete. 2024-04-17T08:02:59.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.641+0000 7f6ed3e00640 1 Processor -- start 2024-04-17T08:02:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3e00640 1 -- start start 2024-04-17T08:02:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed4103fe0 0x7f6ed419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed41051d0 0x7f6ed419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4105b90 0x7f6ed41a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed410fcc0 con 0x7f6ed4103fe0 2024-04-17T08:02:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed410fb40 con 0x7f6ed41051d0 2024-04-17T08:02:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed410fe40 con 0x7f6ed4105b90 2024-04-17T08:02:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4105b90 0x7f6ed41a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed4103fe0 0x7f6ed419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4105b90 0x7f6ed41a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50352/0 (socket says 172.21.15.46:50352) 2024-04-17T08:02:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed3400640 1 -- 172.21.15.46:0/3191075875 learned_addr learned my addr 172.21.15.46:0/3191075875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:02:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.642+0000 7f6ed2000640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed41051d0 0x7f6ed419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ed3400640 1 -- 172.21.15.46:0/3191075875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed41051d0 msgr2=0x7f6ed419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ed3400640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed41051d0 0x7f6ed419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ed3400640 1 -- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed4103fe0 msgr2=0x7f6ed419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ed3400640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed4103fe0 0x7f6ed419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ed3400640 1 -- 172.21.15.46:0/3191075875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed41a1f60 con 0x7f6ed4105b90 2024-04-17T08:02:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ed3400640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4105b90 0x7f6ed41a18f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00eb10 tx=0x7f6ecc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ecb400640 1 -- 172.21.15.46:0/3191075875 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004b20 con 0x7f6ed4105b90 2024-04-17T08:02:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed41a2250 con 0x7f6ed4105b90 2024-04-17T08:02:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ecb400640 1 -- 172.21.15.46:0/3191075875 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc004cc0 con 0x7f6ed4105b90 2024-04-17T08:02:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.643+0000 7f6ecb400640 1 -- 172.21.15.46:0/3191075875 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc010760 con 0x7f6ed4105b90 2024-04-17T08:02:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.644+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed41a27f0 con 0x7f6ed4105b90 2024-04-17T08:02:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.644+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed410b350 con 0x7f6ed4105b90 2024-04-17T08:02:59.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.647+0000 7f6ecb400640 1 -- 172.21.15.46:0/3191075875 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6ecc005020 con 0x7f6ed4105b90 2024-04-17T08:02:59.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.648+0000 7f6ecb400640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6eb80a3bb0 0x7f6eb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:02:59.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.648+0000 7f6ecb400640 1 -- 172.21.15.46:0/3191075875 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f6ecc0d0e80 con 0x7f6ed4105b90 2024-04-17T08:02:59.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.648+0000 7f6ed2a00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6eb80a3bb0 0x7f6eb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:02:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.649+0000 7f6ed2a00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6eb80a3bb0 0x7f6eb80a6070 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402c170 tx=0x7f6ec402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:02:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.653+0000 7f6ecb400640 1 -- 172.21.15.46:0/3191075875 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6ecc09d470 con 0x7f6ed4105b90 2024-04-17T08:02:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.851+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6ed4062b60 con 0x7f6ed4105b90 2024-04-17T08:02:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.852+0000 7f6ecb400640 1 -- 172.21.15.46:0/3191075875 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f6ecc09cd70 con 0x7f6ed4105b90 2024-04-17T08:02:59.851 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-17T08:02:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6eb80a3bb0 msgr2=0x7f6eb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6eb80a3bb0 0x7f6eb80a6070 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402c170 tx=0x7f6ec402df90 comp rx=0 tx=0).stop 2024-04-17T08:02:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4105b90 msgr2=0x7f6ed41a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:02:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4105b90 0x7f6ed41a18f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00eb10 tx=0x7f6ecc00c2d0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 shutdown_connections 2024-04-17T08:02:59.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f6eb80a3bb0 0x7f6eb80a6070 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed4105b90 0x7f6ed41a18f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed41051d0 0x7f6ed419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/3191075875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ed4103fe0 0x7f6ed419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:02:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 >> 172.21.15.46:0/3191075875 conn(0x7f6ed40ff570 msgr2=0x7f6ed4108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:02:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.854+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 shutdown_connections 2024-04-17T08:02:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:02:59.855+0000 7f6ed3e00640 1 -- 172.21.15.46:0/3191075875 wait complete. 2024-04-17T08:02:59.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T08:03:00.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 -- 172.21.15.46:0/3551842766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed81051d0 msgr2=0x7efed8105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:00.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 --2- 172.21.15.46:0/3551842766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed81051d0 0x7efed8105650 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efec8009900 tx=0x7efec802bc40 comp rx=0 tx=0).stop 2024-04-17T08:03:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 -- 172.21.15.46:0/3551842766 shutdown_connections 2024-04-17T08:03:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 --2- 172.21.15.46:0/3551842766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8105b90 0x7efed810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 --2- 172.21.15.46:0/3551842766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed81051d0 0x7efed8105650 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 --2- 172.21.15.46:0/3551842766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed8103fe0 0x7efed8104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 -- 172.21.15.46:0/3551842766 >> 172.21.15.46:0/3551842766 conn(0x7efed80ff570 msgr2=0x7efed8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 -- 172.21.15.46:0/3551842766 shutdown_connections 2024-04-17T08:03:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.089+0000 7efedf200640 1 -- 172.21.15.46:0/3551842766 wait complete. 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.090+0000 7efedf200640 1 Processor -- start 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedf200640 1 -- start start 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedf200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8103fe0 0x7efed819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed81051d0 0x7efed819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8105b90 0x7efed81a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efed810fc70 con 0x7efed8105b90 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedf200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efed810faf0 con 0x7efed8103fe0 2024-04-17T08:03:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efed810fdf0 con 0x7efed81051d0 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8103fe0 0x7efed819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efede800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8105b90 0x7efed81a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedde00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8103fe0 0x7efed819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36670/0 (socket says 172.21.15.46:36670) 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efede800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8105b90 0x7efed81a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39506/0 (socket says 172.21.15.46:39506) 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedde00640 1 -- 172.21.15.46:0/693018840 learned_addr learned my addr 172.21.15.46:0/693018840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.091+0000 7efedd400640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed81051d0 0x7efed819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedde00640 1 -- 172.21.15.46:0/693018840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed81051d0 msgr2=0x7efed819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedde00640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed81051d0 0x7efed819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedde00640 1 -- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8105b90 msgr2=0x7efed81a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedde00640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8105b90 0x7efed81a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedde00640 1 -- 172.21.15.46:0/693018840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed81a1f10 con 0x7efed8103fe0 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedd400640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed81051d0 0x7efed819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedde00640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8103fe0 0x7efed819a2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efecc00e9c0 tx=0x7efecc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efed2a00640 1 -- 172.21.15.46:0/693018840 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00cd80 con 0x7efed8103fe0 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed81a2200 con 0x7efed8103fe0 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efed2a00640 1 -- 172.21.15.46:0/693018840 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efecc0035f0 con 0x7efed8103fe0 2024-04-17T08:03:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.092+0000 7efed2a00640 1 -- 172.21.15.46:0/693018840 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc010690 con 0x7efed8103fe0 2024-04-17T08:03:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.093+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed81a2720 con 0x7efed8103fe0 2024-04-17T08:03:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.094+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed810b300 con 0x7efed8103fe0 2024-04-17T08:03:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.096+0000 7efed2a00640 1 -- 172.21.15.46:0/693018840 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efecc0037a0 con 0x7efed8103fe0 2024-04-17T08:03:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.097+0000 7efed2a00640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efec00a3bb0 0x7efec00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.098+0000 7efed2a00640 1 -- 172.21.15.46:0/693018840 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7efecc0d0ca0 con 0x7efed8103fe0 2024-04-17T08:03:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.098+0000 7efedd400640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efec00a3bb0 0x7efec00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.098+0000 7efedd400640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efec00a3bb0 0x7efec00a6070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efec8006010 tx=0x7efec802c8b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.100+0000 7efed2a00640 1 -- 172.21.15.46:0/693018840 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efecc09d290 con 0x7efed8103fe0 2024-04-17T08:03:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.316+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efed802d5d0 con 0x7efed8103fe0 2024-04-17T08:03:00.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.317+0000 7efed2a00640 1 -- 172.21.15.46:0/693018840 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+585 (secure 0 0 0) 0x7efecc09cb90 con 0x7efed8103fe0 2024-04-17T08:03:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/461113542 2024-04-18T07:47:21.351556+0000 2024-04-17T08:03:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/461113542 2024-04-18T07:47:21.351556+0000 2024-04-17T08:03:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/663006341 2024-04-18T07:47:21.351556+0000 2024-04-17T08:03:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/63594551 2024-04-18T07:46:45.851840+0000 2024-04-17T08:03:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/61074341 2024-04-18T07:47:21.351556+0000 2024-04-17T08:03:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1435664148 2024-04-18T07:46:45.851840+0000 2024-04-17T08:03:00.317 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1911895676 2024-04-18T07:47:21.351556+0000 2024-04-17T08:03:00.317 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/4094803685 2024-04-18T07:46:45.851840+0000 2024-04-17T08:03:00.317 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2289010020 2024-04-18T07:46:45.851840+0000 2024-04-17T08:03:00.317 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/4094803685 2024-04-18T07:46:45.851840+0000 2024-04-17T08:03:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:listed 10 entries 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efec00a3bb0 msgr2=0x7efec00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efec00a3bb0 0x7efec00a6070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efec8006010 tx=0x7efec802c8b0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8103fe0 msgr2=0x7efed819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8103fe0 0x7efed819a2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efecc00e9c0 tx=0x7efecc00ee80 comp rx=0 tx=0).stop 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 shutdown_connections 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7efec00a3bb0 0x7efec00a6070 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed8105b90 0x7efed81a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efed81051d0 0x7efed819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 --2- 172.21.15.46:0/693018840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed8103fe0 0x7efed819a2f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:00.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 >> 172.21.15.46:0/693018840 conn(0x7efed80ff570 msgr2=0x7efed81084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:00.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 shutdown_connections 2024-04-17T08:03:00.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:00.320+0000 7efedf200640 1 -- 172.21.15.46:0/693018840 wait complete. 2024-04-17T08:03:00.339 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-17T08:03:00.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:03:00.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-17T08:03:00.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:00 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2313320135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T08:03:00.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:00 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3191075875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T08:03:00.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:00 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2313320135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T08:03:00.483 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:00 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3191075875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T08:03:00.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-17T08:03:00.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:03:00.488 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T08:03:00.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:00 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2313320135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T08:03:00.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:00 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3191075875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T08:03:00.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:03:00.571 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-17T08:03:00.620 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T08:03:00.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:03:00.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T08:03:00.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T08:03:00.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-17T08:03:00.845 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-17T08:03:00.856 INFO:tasks.cephadm:Teardown begin 2024-04-17T08:03:00.857 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T08:03:00.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T08:03:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 -- 172.21.15.46:0/523477968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 msgr2=0x7f2850110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 --2- 172.21.15.46:0/523477968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f2850110a20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2848009ca0 tx=0x7f284802c100 comp rx=0 tx=0).stop 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 -- 172.21.15.46:0/523477968 shutdown_connections 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 --2- 172.21.15.46:0/523477968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f2850110a20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 --2- 172.21.15.46:0/523477968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28501097d0 0x7f2850109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 --2- 172.21.15.46:0/523477968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28501085e0 0x7f2850108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 -- 172.21.15.46:0/523477968 >> 172.21.15.46:0/523477968 conn(0x7f28500775b0 msgr2=0x7f28500799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 -- 172.21.15.46:0/523477968 shutdown_connections 2024-04-17T08:03:01.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.050+0000 7f2857800640 1 -- 172.21.15.46:0/523477968 wait complete. 2024-04-17T08:03:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.051+0000 7f2857800640 1 Processor -- start 2024-04-17T08:03:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2857800640 1 -- start start 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2857800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28501085e0 0x7f285019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2857800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28501097d0 0x7f285019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2857800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f28501a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2857800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2850114260 con 0x7f28501097d0 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2857800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28501140e0 con 0x7f285010a190 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2857800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f28501143e0 con 0x7f28501085e0 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2847e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28501097d0 0x7f285019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2855800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f28501a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2854e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28501085e0 0x7f285019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2855800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f28501a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36688/0 (socket says 172.21.15.46:36688) 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2855800640 1 -- 172.21.15.46:0/1137787407 learned_addr learned my addr 172.21.15.46:0/1137787407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.052+0000 7f2847e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28501097d0 0x7f285019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39526/0 (socket says 172.21.15.46:39526) 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f2855800640 1 -- 172.21.15.46:0/1137787407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28501085e0 msgr2=0x7f285019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f2855800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28501085e0 0x7f285019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f2855800640 1 -- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28501097d0 msgr2=0x7f285019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f2855800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28501097d0 0x7f285019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f2855800640 1 -- 172.21.15.46:0/1137787407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28501a6490 con 0x7f285010a190 2024-04-17T08:03:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f2855800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f28501a5e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2848037040 tx=0x7f2848035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f284e000640 1 -- 172.21.15.46:0/1137787407 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848039040 con 0x7f285010a190 2024-04-17T08:03:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f284e000640 1 -- 172.21.15.46:0/1137787407 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2848044070 con 0x7f285010a190 2024-04-17T08:03:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.053+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28501a6720 con 0x7f285010a190 2024-04-17T08:03:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.054+0000 7f284e000640 1 -- 172.21.15.46:0/1137787407 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848004500 con 0x7f285010a190 2024-04-17T08:03:01.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.054+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28501a6c60 con 0x7f285010a190 2024-04-17T08:03:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.055+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285010f8a0 con 0x7f285010a190 2024-04-17T08:03:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.057+0000 7f284e000640 1 -- 172.21.15.46:0/1137787407 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f28480046a0 con 0x7f285010a190 2024-04-17T08:03:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.058+0000 7f284e000640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f28340a3c30 0x7f28340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.058+0000 7f284e000640 1 -- 172.21.15.46:0/1137787407 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f28480f2ec0 con 0x7f285010a190 2024-04-17T08:03:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.058+0000 7f2854e00640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f28340a3c30 0x7f28340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.059+0000 7f2854e00640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f28340a3c30 0x7f28340a60f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2850109630 tx=0x7f284000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.060+0000 7f284e000640 1 -- 172.21.15.46:0/1137787407 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f28480bf4b0 con 0x7f285010a190 2024-04-17T08:03:01.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.311+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f285002d790 con 0x7f285010a190 2024-04-17T08:03:01.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.313+0000 7f284e000640 1 -- 172.21.15.46:0/1137787407 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11875 (secure 0 0 0) 0x7f285002d790 con 0x7f285010a190 2024-04-17T08:03:01.312 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:03:01.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T08:02:04.995472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T08:03:01.312 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-17T08:03:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.315+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f28340a3c30 msgr2=0x7f28340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.315+0000 7f2857800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f28340a3c30 0x7f28340a60f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2850109630 tx=0x7f284000b390 comp rx=0 tx=0).stop 2024-04-17T08:03:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.315+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 msgr2=0x7f28501a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.315+0000 7f2857800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f28501a5e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2848037040 tx=0x7f2848035d90 comp rx=0 tx=0).stop 2024-04-17T08:03:01.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.315+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 shutdown_connections 2024-04-17T08:03:01.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.316+0000 7f2857800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f28340a3c30 0x7f28340a60f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.316+0000 7f2857800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f285010a190 0x7f28501a5e20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.316+0000 7f2857800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28501097d0 0x7f285019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.316+0000 7f2857800640 1 --2- 172.21.15.46:0/1137787407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28501085e0 0x7f285019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.316+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 >> 172.21.15.46:0/1137787407 conn(0x7f28500775b0 msgr2=0x7f285010ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.316+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 shutdown_connections 2024-04-17T08:03:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.316+0000 7f2857800640 1 -- 172.21.15.46:0/1137787407 wait complete. 2024-04-17T08:03:01.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T08:03:01.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:01 smithi053 ceph-mon[36437]: pgmap v552: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-17T08:03:01.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:01 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/693018840' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T08:03:01.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 -- 172.21.15.46:0/3019631614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc102920 msgr2=0x7f34dc104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 --2- 172.21.15.46:0/3019631614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc102920 0x7f34dc104d10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009900 tx=0x7f34cc02bc40 comp rx=0 tx=0).stop 2024-04-17T08:03:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 -- 172.21.15.46:0/3019631614 shutdown_connections 2024-04-17T08:03:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 --2- 172.21.15.46:0/3019631614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34dc107b80 0x7f34dc109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 --2- 172.21.15.46:0/3019631614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34dc105250 0x7f34dc107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 --2- 172.21.15.46:0/3019631614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc102920 0x7f34dc104d10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 -- 172.21.15.46:0/3019631614 >> 172.21.15.46:0/3019631614 conn(0x7f34dc0fc330 msgr2=0x7f34dc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 -- 172.21.15.46:0/3019631614 shutdown_connections 2024-04-17T08:03:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.549+0000 7f34e0c00640 1 -- 172.21.15.46:0/3019631614 wait complete. 2024-04-17T08:03:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.550+0000 7f34e0c00640 1 Processor -- start 2024-04-17T08:03:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34e0c00640 1 -- start start 2024-04-17T08:03:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34e0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34dc102920 0x7f34dc102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34e0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc105250 0x7f34dc1008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34e0c00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34dc107b80 0x7f34dc100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34e0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc10db20 con 0x7f34dc105250 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34e0c00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc10d9a0 con 0x7f34dc107b80 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34e0c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc10dca0 con 0x7f34dc102920 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc105250 0x7f34dc1008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc105250 0x7f34dc1008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39536/0 (socket says 172.21.15.46:39536) 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34daa00640 1 -- 172.21.15.46:0/2352713434 learned_addr learned my addr 172.21.15.46:0/2352713434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34db400640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34dc102920 0x7f34dc102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.551+0000 7f34dbe00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34dc107b80 0x7f34dc100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34daa00640 1 -- 172.21.15.46:0/2352713434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34dc102920 msgr2=0x7f34dc102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34daa00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34dc102920 0x7f34dc102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34daa00640 1 -- 172.21.15.46:0/2352713434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34dc107b80 msgr2=0x7f34dc100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34daa00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34dc107b80 0x7f34dc100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34daa00640 1 -- 172.21.15.46:0/2352713434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc1016a0 con 0x7f34dc105250 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34db400640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34dc102920 0x7f34dc102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34daa00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc105250 0x7f34dc1008d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005fd0 tx=0x7f34c800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34d3e00640 1 -- 172.21.15.46:0/2352713434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8017070 con 0x7f34dc105250 2024-04-17T08:03:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc1a22f0 con 0x7f34dc105250 2024-04-17T08:03:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34d3e00640 1 -- 172.21.15.46:0/2352713434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c800de50 con 0x7f34dc105250 2024-04-17T08:03:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.552+0000 7f34d3e00640 1 -- 172.21.15.46:0/2352713434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8003e00 con 0x7f34dc105250 2024-04-17T08:03:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.553+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc1a2840 con 0x7f34dc105250 2024-04-17T08:03:01.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:01 smithi062 ceph-mon[36507]: pgmap v552: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-17T08:03:01.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:01 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/693018840' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T08:03:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.554+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc0fde20 con 0x7f34dc105250 2024-04-17T08:03:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.557+0000 7f34d3e00640 1 -- 172.21.15.46:0/2352713434 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f34c8003fa0 con 0x7f34dc105250 2024-04-17T08:03:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.558+0000 7f34d3e00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34c00a3ba0 0x7f34c00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.558+0000 7f34d3e00640 1 -- 172.21.15.46:0/2352713434 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f34c80d0400 con 0x7f34dc105250 2024-04-17T08:03:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.559+0000 7f34db400640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34c00a3ba0 0x7f34c00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:01.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.559+0000 7f34db400640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34c00a3ba0 0x7f34c00a6060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f34cc02c7c0 tx=0x7f34cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:01.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.564+0000 7f34d3e00640 1 -- 172.21.15.46:0/2352713434 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f34c8020030 con 0x7f34dc105250 2024-04-17T08:03:01.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:01 smithi046 ceph-mon[32038]: pgmap v552: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-17T08:03:01.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:01 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/693018840' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T08:03:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.812+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34dc02d5d0 con 0x7f34dc105250 2024-04-17T08:03:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.814+0000 7f34d3e00640 1 -- 172.21.15.46:0/2352713434 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11875 (secure 0 0 0) 0x7f34c809c9f0 con 0x7f34dc105250 2024-04-17T08:03:01.813 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:03:01.813 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14379,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1217684239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1217684239},{"type":"v1","addr":"172.21.15.46:6835","nonce":1217684239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14391,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1258153884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1258153884},{"type":"v1","addr":"172.21.15.46:6837","nonce":1258153884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/1599229052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":1599229052},{"type":"v1","addr":"172.21.15.46:6841","nonce":1599229052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1314622841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1314622841},{"type":"v1","addr":"172.21.15.62:6833","nonce":1314622841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2557309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2557309272},{"type":"v1","addr":"172.21.15.53:6833","nonce":2557309272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3940284604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3940284604},{"type":"v1","addr":"172.21.15.53:6835","nonce":3940284604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:55:26.016968+0000","modified":"2024-04-17T08:02:04.995472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24596,"mds_1":24556,"mds_2":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6835/2191814520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2191814520},{"type":"v1","addr":"172.21.15.62:6835","nonce":2191814520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/956269488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":956269488},{"type":"v1","addr":"172.21.15.62:6837","nonce":956269488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/665634381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":665634381},{"type":"v1","addr":"172.21.15.62:6839","nonce":665634381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.53:6837/2183592848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":2183592848},{"type":"v1","addr":"172.21.15.53:6837","nonce":2183592848}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.46:6839/137300158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":137300158},{"type":"v1","addr":"172.21.15.46:6839","nonce":137300158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/262156089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":262156089},{"type":"v1","addr":"172.21.15.53:6839","nonce":262156089}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596,24542,24556]},"id":1}]} 2024-04-17T08:03:01.814 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-04-17T08:03:01.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34c00a3ba0 msgr2=0x7f34c00a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34c00a3ba0 0x7f34c00a6060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f34cc02c7c0 tx=0x7f34cc005fd0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc105250 msgr2=0x7f34dc1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc105250 0x7f34dc1008d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005fd0 tx=0x7f34c800dd20 comp rx=0 tx=0).stop 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 shutdown_connections 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f34c00a3ba0 0x7f34c00a6060 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34dc107b80 0x7f34dc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34dc105250 0x7f34dc1008d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 --2- 172.21.15.46:0/2352713434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34dc102920 0x7f34dc102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:01.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 >> 172.21.15.46:0/2352713434 conn(0x7f34dc0fc330 msgr2=0x7f34dc1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 shutdown_connections 2024-04-17T08:03:01.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:01.817+0000 7f34e0c00640 1 -- 172.21.15.46:0/2352713434 wait complete. 2024-04-17T08:03:01.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T08:03:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 -- 172.21.15.46:0/545374800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 msgr2=0x7f481c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 --2- 172.21.15.46:0/545374800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c108a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f480c009900 tx=0x7f480c02bc40 comp rx=0 tx=0).stop 2024-04-17T08:03:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 -- 172.21.15.46:0/545374800 shutdown_connections 2024-04-17T08:03:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 --2- 172.21.15.46:0/545374800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c10a190 0x7f481c110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 --2- 172.21.15.46:0/545374800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c1097d0 0x7f481c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 --2- 172.21.15.46:0/545374800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c108a00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 -- 172.21.15.46:0/545374800 >> 172.21.15.46:0/545374800 conn(0x7f481c0775b0 msgr2=0x7f481c0799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 -- 172.21.15.46:0/545374800 shutdown_connections 2024-04-17T08:03:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.062+0000 7f4823000640 1 -- 172.21.15.46:0/545374800 wait complete. 2024-04-17T08:03:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.063+0000 7f4823000640 1 Processor -- start 2024-04-17T08:03:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4823000640 1 -- start start 2024-04-17T08:03:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4823000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4823000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c1097d0 0x7f481c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4823000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c10a190 0x7f481c1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4823000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1142a0 con 0x7f481c1097d0 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4823000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f481c114120 con 0x7f481c1085e0 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4823000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f481c114420 con 0x7f481c10a190 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f481be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f481be00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36716/0 (socket says 172.21.15.46:36716) 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f481be00640 1 -- 172.21.15.46:0/4064278798 learned_addr learned my addr 172.21.15.46:0/4064278798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4813400640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c1097d0 0x7f481c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.064+0000 7f4821000640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c10a190 0x7f481c1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f481be00640 1 -- 172.21.15.46:0/4064278798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c10a190 msgr2=0x7f481c1a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f481be00640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c10a190 0x7f481c1a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f481be00640 1 -- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c1097d0 msgr2=0x7f481c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f481be00640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c1097d0 0x7f481c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f481be00640 1 -- 172.21.15.46:0/4064278798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c1a6530 con 0x7f481c1085e0 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f481be00640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c19e910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f480c02efd0 tx=0x7f480c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f4819600640 1 -- 172.21.15.46:0/4064278798 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c041070 con 0x7f481c1085e0 2024-04-17T08:03:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f4819600640 1 -- 172.21.15.46:0/4064278798 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480c0044d0 con 0x7f481c1085e0 2024-04-17T08:03:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c1a67c0 con 0x7f481c1085e0 2024-04-17T08:03:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f4819600640 1 -- 172.21.15.46:0/4064278798 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c034440 con 0x7f481c1085e0 2024-04-17T08:03:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.065+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a6cd0 con 0x7f481c1085e0 2024-04-17T08:03:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.066+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c10f900 con 0x7f481c1085e0 2024-04-17T08:03:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.069+0000 7f4819600640 1 -- 172.21.15.46:0/4064278798 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f480c044070 con 0x7f481c1085e0 2024-04-17T08:03:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.070+0000 7f4819600640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48000a3c30 0x7f48000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.070+0000 7f4819600640 1 -- 172.21.15.46:0/4064278798 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f480c0f2670 con 0x7f481c1085e0 2024-04-17T08:03:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.070+0000 7f4813400640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48000a3c30 0x7f48000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.071+0000 7f4813400640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48000a3c30 0x7f48000a60f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f481c19fe20 tx=0x7f4804009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.072+0000 7f4819600640 1 -- 172.21.15.46:0/4064278798 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f480c0be610 con 0x7f481c1085e0 2024-04-17T08:03:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.283+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f481c02d7a0 con 0x7f481c1085e0 2024-04-17T08:03:02.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.285+0000 7f4819600640 1 -- 172.21.15.46:0/4064278798 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24609 (secure 0 0 0) 0x7f480c0c2910 con 0x7f481c1085e0 2024-04-17T08:03:02.285 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-17T08:03:02.285 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"67cd6c06-fc8e-11ee-bc91-c7b262605968","created":"2024-04-17T07:46:11.976719+0000","modified":"2024-04-17T07:57:22.972449+0000","last_up_change":"2024-04-17T07:54:26.814789+0000","last_in_change":"2024-04-17T07:54:06.728371+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:50:07.527032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:55:24.035234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:55:24.942051+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:55:28.582645+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"437464e9-4b20-4de4-9782-89a0daec7f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6803","nonce":3982963402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6805","nonce":3982963402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6809","nonce":3982963402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3982963402},{"type":"v1","addr":"172.21.15.46:6807","nonce":3982963402}]},"public_addr":"172.21.15.46:6803/3982963402","cluster_addr":"172.21.15.46:6805/3982963402","heartbeat_back_addr":"172.21.15.46:6809/3982963402","heartbeat_front_addr":"172.21.15.46:6807/3982963402","state":["exists","up"]},{"osd":1,"uuid":"d0511b8c-f523-4678-9683-ceaa6a1e65d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6801","nonce":445046635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6803","nonce":445046635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6807","nonce":445046635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":445046635},{"type":"v1","addr":"172.21.15.53:6805","nonce":445046635}]},"public_addr":"172.21.15.53:6801/445046635","cluster_addr":"172.21.15.53:6803/445046635","heartbeat_back_addr":"172.21.15.53:6807/445046635","heartbeat_front_addr":"172.21.15.53:6805/445046635","state":["exists","up"]},{"osd":2,"uuid":"d3c4feb5-5f8c-46d7-8e8c-dc32f24ad246","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6801","nonce":3705420480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6803","nonce":3705420480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6807","nonce":3705420480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3705420480},{"type":"v1","addr":"172.21.15.62:6805","nonce":3705420480}]},"public_addr":"172.21.15.62:6801/3705420480","cluster_addr":"172.21.15.62:6803/3705420480","heartbeat_back_addr":"172.21.15.62:6807/3705420480","heartbeat_front_addr":"172.21.15.62:6805/3705420480","state":["exists","up"]},{"osd":3,"uuid":"398d84f2-695b-4446-872d-55ba26ab8c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6811","nonce":24338866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6813","nonce":24338866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6817","nonce":24338866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":24338866},{"type":"v1","addr":"172.21.15.46:6815","nonce":24338866}]},"public_addr":"172.21.15.46:6811/24338866","cluster_addr":"172.21.15.46:6813/24338866","heartbeat_back_addr":"172.21.15.46:6817/24338866","heartbeat_front_addr":"172.21.15.46:6815/24338866","state":["exists","up"]},{"osd":4,"uuid":"5fb57543-bfba-422e-9306-4b1b50a98c0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6809","nonce":3909086430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6811","nonce":3909086430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6815","nonce":3909086430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3909086430},{"type":"v1","addr":"172.21.15.53:6813","nonce":3909086430}]},"public_addr":"172.21.15.53:6809/3909086430","cluster_addr":"172.21.15.53:6811/3909086430","heartbeat_back_addr":"172.21.15.53:6815/3909086430","heartbeat_front_addr":"172.21.15.53:6813/3909086430","state":["exists","up"]},{"osd":5,"uuid":"26424544-5d91-45e4-8e62-00945c375bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6809","nonce":2113355560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6811","nonce":2113355560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6815","nonce":2113355560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2113355560},{"type":"v1","addr":"172.21.15.62:6813","nonce":2113355560}]},"public_addr":"172.21.15.62:6809/2113355560","cluster_addr":"172.21.15.62:6811/2113355560","heartbeat_back_addr":"172.21.15.62:6815/2113355560","heartbeat_front_addr":"172.21.15.62:6813/2113355560","state":["exists","up"]},{"osd":6,"uuid":"720d8bdb-9a6a-4fcc-bda7-b575a895ea17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6819","nonce":1575836567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6821","nonce":1575836567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6825","nonce":1575836567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":1575836567},{"type":"v1","addr":"172.21.15.46:6823","nonce":1575836567}]},"public_addr":"172.21.15.46:6819/1575836567","cluster_addr":"172.21.15.46:6821/1575836567","heartbeat_back_addr":"172.21.15.46:6825/1575836567","heartbeat_front_addr":"172.21.15.46:6823/1575836567","state":["exists","up"]},{"osd":7,"uuid":"7b7d7a00-52ea-47ab-b844-d5a192861516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6817","nonce":3581494235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6819","nonce":3581494235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6823","nonce":3581494235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3581494235},{"type":"v1","addr":"172.21.15.53:6821","nonce":3581494235}]},"public_addr":"172.21.15.53:6817/3581494235","cluster_addr":"172.21.15.53:6819/3581494235","heartbeat_back_addr":"172.21.15.53:6823/3581494235","heartbeat_front_addr":"172.21.15.53:6821/3581494235","state":["exists","up"]},{"osd":8,"uuid":"0f936f20-269a-496b-af26-a962f09c26a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6817","nonce":520671268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6819","nonce":520671268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6823","nonce":520671268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":520671268},{"type":"v1","addr":"172.21.15.62:6821","nonce":520671268}]},"public_addr":"172.21.15.62:6817/520671268","cluster_addr":"172.21.15.62:6819/520671268","heartbeat_back_addr":"172.21.15.62:6823/520671268","heartbeat_front_addr":"172.21.15.62:6821/520671268","state":["exists","up"]},{"osd":9,"uuid":"31f44d01-ea3b-4b7a-a763-8d39e757395b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6827","nonce":1655693057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6829","nonce":1655693057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6833","nonce":1655693057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1655693057},{"type":"v1","addr":"172.21.15.46:6831","nonce":1655693057}]},"public_addr":"172.21.15.46:6827/1655693057","cluster_addr":"172.21.15.46:6829/1655693057","heartbeat_back_addr":"172.21.15.46:6833/1655693057","heartbeat_front_addr":"172.21.15.46:6831/1655693057","state":["exists","up"]},{"osd":10,"uuid":"93b1b906-94d1-4af6-8ab1-b69157e8185c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6825","nonce":795360717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6827","nonce":795360717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6831","nonce":795360717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":795360717},{"type":"v1","addr":"172.21.15.53:6829","nonce":795360717}]},"public_addr":"172.21.15.53:6825/795360717","cluster_addr":"172.21.15.53:6827/795360717","heartbeat_back_addr":"172.21.15.53:6831/795360717","heartbeat_front_addr":"172.21.15.53:6829/795360717","state":["exists","up"]},{"osd":11,"uuid":"aaf98df8-8674-48c4-87d5-531ef5a44098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6825","nonce":359287884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6827","nonce":359287884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6831","nonce":359287884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":359287884},{"type":"v1","addr":"172.21.15.62:6829","nonce":359287884}]},"public_addr":"172.21.15.62:6825/359287884","cluster_addr":"172.21.15.62:6827/359287884","heartbeat_back_addr":"172.21.15.62:6831/359287884","heartbeat_front_addr":"172.21.15.62:6829/359287884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:13.320742+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:49:39.022212+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:03.995941+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:50:37.010146+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:03.436200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:51:29.309112+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:03.497279+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:30.873810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:52:56.472260+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:32.124519+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:53:59.297196+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:54:24.918259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/461113542":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/663006341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/63594551":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/61074341":"2024-04-18T07:47:21.351556+0000","172.21.15.46:0/1435664148":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/1911895676":"2024-04-18T07:47:21.351556+0000","172.21.15.46:6800/4094803685":"2024-04-18T07:46:45.851840+0000","172.21.15.46:0/2289010020":"2024-04-18T07:46:45.851840+0000","172.21.15.46:6801/4094803685":"2024-04-18T07:46:45.851840+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T08:03:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48000a3c30 msgr2=0x7f48000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48000a3c30 0x7f48000a60f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f481c19fe20 tx=0x7f4804009500 comp rx=0 tx=0).stop 2024-04-17T08:03:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 msgr2=0x7f481c19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c19e910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f480c02efd0 tx=0x7f480c005d50 comp rx=0 tx=0).stop 2024-04-17T08:03:02.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 shutdown_connections 2024-04-17T08:03:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f48000a3c30 0x7f48000a60f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f481c10a190 0x7f481c1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481c1097d0 0x7f481c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 --2- 172.21.15.46:0/4064278798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f481c1085e0 0x7f481c19e910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 >> 172.21.15.46:0/4064278798 conn(0x7f481c0775b0 msgr2=0x7f481c10cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 shutdown_connections 2024-04-17T08:03:02.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.288+0000 7f4823000640 1 -- 172.21.15.46:0/4064278798 wait complete. 2024-04-17T08:03:02.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T08:03:02.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1137787407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:03:02.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/2352713434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:03:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 -- 172.21.15.46:0/453523008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241053b0 msgr2=0x7f6624105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 --2- 172.21.15.46:0/453523008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241053b0 0x7f6624105830 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f660c009ca0 tx=0x7f660c02c100 comp rx=0 tx=0).stop 2024-04-17T08:03:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 -- 172.21.15.46:0/453523008 shutdown_connections 2024-04-17T08:03:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 --2- 172.21.15.46:0/453523008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6624105d70 0x7f662410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 --2- 172.21.15.46:0/453523008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241053b0 0x7f6624105830 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 --2- 172.21.15.46:0/453523008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66241041c0 0x7f66241045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 -- 172.21.15.46:0/453523008 >> 172.21.15.46:0/453523008 conn(0x7f66240ff750 msgr2=0x7f6624101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 -- 172.21.15.46:0/453523008 shutdown_connections 2024-04-17T08:03:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.535+0000 7f662a200640 1 -- 172.21.15.46:0/453523008 wait complete. 2024-04-17T08:03:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 Processor -- start 2024-04-17T08:03:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 -- start start 2024-04-17T08:03:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241041c0 0x7f662419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66241053b0 0x7f662419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6624105d70 0x7f66241a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f662410fe50 con 0x7f66241053b0 2024-04-17T08:03:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f662410fcd0 con 0x7f66241041c0 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f662a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f662410ffd0 con 0x7f6624105d70 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f6622a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66241053b0 0x7f662419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.537+0000 7f6623400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241041c0 0x7f662419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6623400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241041c0 0x7f662419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.46:36732/0 (socket says 172.21.15.46:36732) 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6623400640 1 -- 172.21.15.46:0/4033862709 learned_addr learned my addr 172.21.15.46:0/4033862709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6623e00640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6624105d70 0x7f66241a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6622a00640 1 -- 172.21.15.46:0/4033862709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6624105d70 msgr2=0x7f66241a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6622a00640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6624105d70 0x7f66241a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6622a00640 1 -- 172.21.15.46:0/4033862709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241041c0 msgr2=0x7f662419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6622a00640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241041c0 0x7f662419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6622a00640 1 -- 172.21.15.46:0/4033862709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66241a20d0 con 0x7f66241053b0 2024-04-17T08:03:02.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6623400640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241041c0 0x7f662419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:03:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.538+0000 7f6622a00640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66241053b0 0x7f662419a9f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f660c02c640 tx=0x7f660c035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.539+0000 7f661be00640 1 -- 172.21.15.46:0/4033862709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c039040 con 0x7f66241053b0 2024-04-17T08:03:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.539+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66241a2360 con 0x7f66241053b0 2024-04-17T08:03:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.539+0000 7f661be00640 1 -- 172.21.15.46:0/4033862709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660c003970 con 0x7f66241053b0 2024-04-17T08:03:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.539+0000 7f661be00640 1 -- 172.21.15.46:0/4033862709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c004330 con 0x7f66241053b0 2024-04-17T08:03:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.539+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66241a28a0 con 0x7f66241053b0 2024-04-17T08:03:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.540+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662410b4e0 con 0x7f66241053b0 2024-04-17T08:03:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.542+0000 7f661be00640 1 -- 172.21.15.46:0/4033862709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f660c046030 con 0x7f66241053b0 2024-04-17T08:03:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.544+0000 7f661be00640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66040a3c30 0x7f66040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:02.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.544+0000 7f661be00640 1 -- 172.21.15.46:0/4033862709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f660c0f28a0 con 0x7f66241053b0 2024-04-17T08:03:02.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.544+0000 7f6623400640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66040a3c30 0x7f66040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:02.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.545+0000 7f6623400640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66040a3c30 0x7f66040a60f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f66140058c0 tx=0x7f6614005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:02.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.547+0000 7f661be00640 1 -- 172.21.15.46:0/4033862709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f660c04d060 con 0x7f66241053b0 2024-04-17T08:03:02.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1137787407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:03:02.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/2352713434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:03:02.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:02 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1137787407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:03:02.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:02 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/2352713434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T08:03:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.800+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f662402d7a0 con 0x7f66241053b0 2024-04-17T08:03:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.806+0000 7f661be00640 1 -- 172.21.15.46:0/4033862709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f660c0bef20 con 0x7f66241053b0 2024-04-17T08:03:02.805 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T08:03:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66040a3c30 msgr2=0x7f66040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66040a3c30 0x7f66040a60f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f66140058c0 tx=0x7f6614005f90 comp rx=0 tx=0).stop 2024-04-17T08:03:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66241053b0 msgr2=0x7f662419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66241053b0 0x7f662419a9f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f660c02c640 tx=0x7f660c035d90 comp rx=0 tx=0).stop 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 shutdown_connections 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f66040a3c30 0x7f66040a60f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6624105d70 0x7f66241a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66241053b0 0x7f662419a9f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 --2- 172.21.15.46:0/4033862709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66241041c0 0x7f662419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 >> 172.21.15.46:0/4033862709 conn(0x7f66240ff750 msgr2=0x7f66241086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 shutdown_connections 2024-04-17T08:03:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:02.809+0000 7f662a200640 1 -- 172.21.15.46:0/4033862709 wait complete. 2024-04-17T08:03:02.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T08:03:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 -- 172.21.15.46:0/3102849383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0101dd0 msgr2=0x7f99d01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 --2- 172.21.15.46:0/3102849383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0101dd0 0x7f99d01021f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009900 tx=0x7f99c002bc40 comp rx=0 tx=0).stop 2024-04-17T08:03:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 -- 172.21.15.46:0/3102849383 shutdown_connections 2024-04-17T08:03:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 --2- 172.21.15.46:0/3102849383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d0103980 0x7f99d010a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 --2- 172.21.15.46:0/3102849383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99d0102fc0 0x7f99d0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 --2- 172.21.15.46:0/3102849383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0101dd0 0x7f99d01021f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 -- 172.21.15.46:0/3102849383 >> 172.21.15.46:0/3102849383 conn(0x7f99d00fd340 msgr2=0x7f99d00ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 -- 172.21.15.46:0/3102849383 shutdown_connections 2024-04-17T08:03:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.038+0000 7f99d5800640 1 -- 172.21.15.46:0/3102849383 wait complete. 2024-04-17T08:03:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.039+0000 7f99d5800640 1 Processor -- start 2024-04-17T08:03:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d5800640 1 -- start start 2024-04-17T08:03:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99d0101dd0 0x7f99d0075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d5800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0102fc0 0x7f99d0074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d0103980 0x7f99d0074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99d010daa0 con 0x7f99d0101dd0 2024-04-17T08:03:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d5800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99d010d920 con 0x7f99d0102fc0 2024-04-17T08:03:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d5800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f99d010dc20 con 0x7f99d0103980 2024-04-17T08:03:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99d0101dd0 0x7f99d0075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99d4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d0103980 0x7f99d0074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99d0101dd0 0x7f99d0075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39604/0 (socket says 172.21.15.46:39604) 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99cf400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0102fc0 0x7f99d0074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.040+0000 7f99cfe00640 1 -- 172.21.15.46:0/1020350663 learned_addr learned my addr 172.21.15.46:0/1020350663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99cf400640 1 -- 172.21.15.46:0/1020350663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d0103980 msgr2=0x7f99d0074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99cf400640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d0103980 0x7f99d0074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99cf400640 1 -- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99d0101dd0 msgr2=0x7f99d0075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99cf400640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99d0101dd0 0x7f99d0075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99cf400640 1 -- 172.21.15.46:0/1020350663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d0075120 con 0x7f99d0102fc0 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99cf400640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0102fc0 0x7f99d0074350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9f0 tx=0x7f99bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99ccc00640 1 -- 172.21.15.46:0/1020350663 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00ccc0 con 0x7f99d0102fc0 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99ccc00640 1 -- 172.21.15.46:0/1020350663 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc010910 con 0x7f99d0102fc0 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99ccc00640 1 -- 172.21.15.46:0/1020350663 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc018960 con 0x7f99d0102fc0 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.041+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d01a2240 con 0x7f99d0102fc0 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.042+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d01a2740 con 0x7f99d0102fc0 2024-04-17T08:03:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.043+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d01090f0 con 0x7f99d0102fc0 2024-04-17T08:03:03.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.045+0000 7f99ccc00640 1 -- 172.21.15.46:0/1020350663 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99bc010ac0 con 0x7f99d0102fc0 2024-04-17T08:03:03.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.047+0000 7f99ccc00640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f99b40a3bb0 0x7f99b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:03.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.047+0000 7f99ccc00640 1 -- 172.21.15.46:0/1020350663 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f99bc0d0c70 con 0x7f99d0102fc0 2024-04-17T08:03:03.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.047+0000 7f99cfe00640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f99b40a3bb0 0x7f99b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:03.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.048+0000 7f99cfe00640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f99b40a3bb0 0x7f99b40a6070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f99c0005ab0 tx=0x7f99c002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:03.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.048+0000 7f99ccc00640 1 -- 172.21.15.46:0/1020350663 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f99bc09cb30 con 0x7f99d0102fc0 2024-04-17T08:03:03.109 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:02 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a[32034]: 2024-04-17T08:03:02.802+0000 7f9d7ec00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T08:03:03.109 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:03:02 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g[73480]: ignoring --setuser ceph since I am not root 2024-04-17T08:03:03.110 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:03:02 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g[73480]: ignoring --setgroup ceph since I am not root 2024-04-17T08:03:03.110 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:03:02 smithi046 ceph-mds[73484]: -- 172.21.15.46:0/770223260 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b948f9da40 con 0x55b948f70c00 2024-04-17T08:03:03.110 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:03:02 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g[73480]: starting mds.g at 2024-04-17T08:03:03.218 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c[67682]: ignoring --setuser ceph since I am not root 2024-04-17T08:03:03.218 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c[67682]: ignoring --setgroup ceph since I am not root 2024-04-17T08:03:03.219 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c[67682]: starting mds.c at 2024-04-17T08:03:03.220 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: ignoring --setuser ceph since I am not root 2024-04-17T08:03:03.220 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: ignoring --setgroup ceph since I am not root 2024-04-17T08:03:03.220 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-mds[67397]: -- 172.21.15.53:0/1678446766 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55732e7b9a40 con 0x55732e78c800 2024-04-17T08:03:03.220 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: starting mds.b at 2024-04-17T08:03:03.220 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i[67994]: ignoring --setuser ceph since I am not root 2024-04-17T08:03:03.220 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i[67994]: ignoring --setgroup ceph since I am not root 2024-04-17T08:03:03.221 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-mds[67998]: -- 172.21.15.62:0/760938418 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e050ede4e0 con 0x55e050eb4c00 2024-04-17T08:03:03.221 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-mds[67998]: -- 172.21.15.62:0/760938418 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e050ee1a40 con 0x55e050eb4c00 2024-04-17T08:03:03.221 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:03:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i[67994]: starting mds.i at 2024-04-17T08:03:03.221 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h[67073]: ignoring --setuser ceph since I am not root 2024-04-17T08:03:03.221 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h[67073]: ignoring --setgroup ceph since I am not root 2024-04-17T08:03:03.221 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-mds[67077]: -- 172.21.15.53:0/2885728698 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56509f4e7a40 con 0x56509f4ba800 2024-04-17T08:03:03.221 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:03:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h[67073]: starting mds.h at 2024-04-17T08:03:03.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.300+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f99d002d820 con 0x7f99d0102fc0 2024-04-17T08:03:03.470 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: pgmap v553: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-17T08:03:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4064278798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:03:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4033862709' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-17T08:03:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-17T08:03:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T08:03:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/4033862709' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-17T08:03:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: osdmap e94: 12 total, 12 up, 12 in 2024-04-17T08:03:03.471 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:03 smithi053 ceph-mon[36437]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-17T08:03:03.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: pgmap v553: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-17T08:03:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4064278798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:03:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4033862709' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-17T08:03:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-17T08:03:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T08:03:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/4033862709' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-17T08:03:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: osdmap e94: 12 total, 12 up, 12 in 2024-04-17T08:03:03.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:03 smithi062 ceph-mon[36507]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-17T08:03:03.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: pgmap v553: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-17T08:03:03.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4064278798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T08:03:03.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4033862709' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-17T08:03:03.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-17T08:03:03.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T08:03:03.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/4033862709' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-17T08:03:03.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: osdmap e94: 12 total, 12 up, 12 in 2024-04-17T08:03:03.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:03 smithi046 ceph-mon[32038]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-17T08:03:03.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.812+0000 7f99ccc00640 1 -- 172.21.15.46:0/1020350663 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f99bc0a0e30 con 0x7f99d0102fc0 2024-04-17T08:03:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.816+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f99b40a3bb0 msgr2=0x7f99b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.816+0000 7f99d5800640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f99b40a3bb0 0x7f99b40a6070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f99c0005ab0 tx=0x7f99c002cf90 comp rx=0 tx=0).stop 2024-04-17T08:03:03.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.816+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0102fc0 msgr2=0x7f99d0074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:03.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.816+0000 7f99d5800640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0102fc0 0x7f99d0074350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9f0 tx=0x7f99bc00eeb0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.817+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 shutdown_connections 2024-04-17T08:03:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.817+0000 7f99d5800640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f99b40a3bb0 0x7f99b40a6070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.817+0000 7f99d5800640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99d0103980 0x7f99d0074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.817+0000 7f99d5800640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d0102fc0 0x7f99d0074350 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.817+0000 7f99d5800640 1 --2- 172.21.15.46:0/1020350663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99d0101dd0 0x7f99d0075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.817+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 >> 172.21.15.46:0/1020350663 conn(0x7f99d00fd340 msgr2=0x7f99d01062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:03.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.818+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 shutdown_connections 2024-04-17T08:03:03.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:03.818+0000 7f99d5800640 1 -- 172.21.15.46:0/1020350663 wait complete. 2024-04-17T08:03:03.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T08:03:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.065+0000 7f931fe00640 1 -- 172.21.15.46:0/490649209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320103950 msgr2=0x7f932010a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.065+0000 7f931fe00640 1 --2- 172.21.15.46:0/490649209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320103950 0x7f932010a1e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9318009ca0 tx=0x7f931802c0f0 comp rx=0 tx=0).stop 2024-04-17T08:03:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.066+0000 7f931fe00640 1 -- 172.21.15.46:0/490649209 shutdown_connections 2024-04-17T08:03:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.066+0000 7f931fe00640 1 --2- 172.21.15.46:0/490649209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320103950 0x7f932010a1e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.066+0000 7f931fe00640 1 --2- 172.21.15.46:0/490649209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9320102f90 0x7f9320103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.066+0000 7f931fe00640 1 --2- 172.21.15.46:0/490649209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9320101da0 0x7f93201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.066+0000 7f931fe00640 1 -- 172.21.15.46:0/490649209 >> 172.21.15.46:0/490649209 conn(0x7f93200fd310 msgr2=0x7f93200ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.066+0000 7f931fe00640 1 -- 172.21.15.46:0/490649209 shutdown_connections 2024-04-17T08:03:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.066+0000 7f931fe00640 1 -- 172.21.15.46:0/490649209 wait complete. 2024-04-17T08:03:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 Processor -- start 2024-04-17T08:03:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 -- start start 2024-04-17T08:03:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320101da0 0x7f9320198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9320102f90 0x7f9320198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9320103950 0x7f932019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f932010da20 con 0x7f9320101da0 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f932010d8a0 con 0x7f9320102f90 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f932010dba0 con 0x7f9320103950 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.067+0000 7f931ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320101da0 0x7f9320198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9320103950 0x7f932019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931e000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9320102f90 0x7f9320198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320101da0 0x7f9320198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39622/0 (socket says 172.21.15.46:39622) 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 -- 172.21.15.46:0/3926274446 learned_addr learned my addr 172.21.15.46:0/3926274446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 -- 172.21.15.46:0/3926274446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9320103950 msgr2=0x7f932019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9320103950 0x7f932019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 -- 172.21.15.46:0/3926274446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9320102f90 msgr2=0x7f9320198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9320102f90 0x7f9320198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 -- 172.21.15.46:0/3926274446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932019fc20 con 0x7f9320101da0 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931e000640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9320102f90 0x7f9320198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T08:03:04.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f931ea00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320101da0 0x7f9320198000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f931000e9c0 tx=0x7f931000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f9324a00640 1 -- 172.21.15.46:0/3926274446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931000cd80 con 0x7f9320101da0 2024-04-17T08:03:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.068+0000 7f9324a00640 1 -- 172.21.15.46:0/3926274446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93100035f0 con 0x7f9320101da0 2024-04-17T08:03:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.069+0000 7f9324a00640 1 -- 172.21.15.46:0/3926274446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310010690 con 0x7f9320101da0 2024-04-17T08:03:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.069+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93200741f0 con 0x7f9320101da0 2024-04-17T08:03:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.069+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9320074740 con 0x7f9320101da0 2024-04-17T08:03:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.070+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9320109060 con 0x7f9320101da0 2024-04-17T08:03:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.072+0000 7f9324a00640 1 -- 172.21.15.46:0/3926274446 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f93100037a0 con 0x7f9320101da0 2024-04-17T08:03:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.074+0000 7f9324a00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93040a3c20 0x7f93040a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.074+0000 7f9324a00640 1 -- 172.21.15.46:0/3926274446 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f93100d0bb0 con 0x7f9320101da0 2024-04-17T08:03:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.074+0000 7f931e000640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93040a3c20 0x7f93040a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.075+0000 7f931e000640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93040a3c20 0x7f93040a60e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f930c0095f0 tx=0x7f930c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.079+0000 7f9324a00640 1 -- 172.21.15.46:0/3926274446 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f931009d0c0 con 0x7f9320101da0 2024-04-17T08:03:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:04.280+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9320075b80 con 0x7f9320101da0 2024-04-17T08:03:04.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/1020350663' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: Cluster is now healthy 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.62:6836/1574346015,v1:172.21.15.62:6839/1574346015] up:boot 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.62:6835/4275374373,v1:172.21.15.62:6838/4275374373] up:boot 2024-04-17T08:03:04.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6836/1138292142,v1:172.21.15.53:6838/1138292142] up:boot 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.62:6834/2985993413,v1:172.21.15.62:6837/2985993413] up:boot 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.53:6837/3566213796,v1:172.21.15.53:6839/3566213796] up:boot 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: mds.? [v2:172.21.15.46:6838/3771802136,v1:172.21.15.46:6839/3771802136] up:boot 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: fsmap 12 up:standby 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T08:03:04.554 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T08:03:04.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T08:03:04.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:04 smithi062 ceph-mon[36507]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T08:03:04.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/1020350663' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T08:03:04.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T08:03:04.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-17T08:03:04.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-17T08:03:04.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: Cluster is now healthy 2024-04-17T08:03:04.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.62:6836/1574346015,v1:172.21.15.62:6839/1574346015] up:boot 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.62:6835/4275374373,v1:172.21.15.62:6838/4275374373] up:boot 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6836/1138292142,v1:172.21.15.53:6838/1138292142] up:boot 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.62:6834/2985993413,v1:172.21.15.62:6837/2985993413] up:boot 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.53:6837/3566213796,v1:172.21.15.53:6839/3566213796] up:boot 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: mds.? [v2:172.21.15.46:6838/3771802136,v1:172.21.15.46:6839/3771802136] up:boot 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: fsmap 12 up:standby 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T08:03:04.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T08:03:04.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T08:03:04.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T08:03:04.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T08:03:04.613 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:04 smithi046 ceph-mon[32038]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T08:03:04.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/1020350663' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T08:03:04.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T08:03:04.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-17T08:03:04.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-17T08:03:04.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: Cluster is now healthy 2024-04-17T08:03:04.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-17T08:03:04.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.62:6836/1574346015,v1:172.21.15.62:6839/1574346015] up:boot 2024-04-17T08:03:04.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.62:6835/4275374373,v1:172.21.15.62:6838/4275374373] up:boot 2024-04-17T08:03:04.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6836/1138292142,v1:172.21.15.53:6838/1138292142] up:boot 2024-04-17T08:03:04.722 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.62:6834/2985993413,v1:172.21.15.62:6837/2985993413] up:boot 2024-04-17T08:03:04.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.53:6837/3566213796,v1:172.21.15.53:6839/3566213796] up:boot 2024-04-17T08:03:04.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: mds.? [v2:172.21.15.46:6838/3771802136,v1:172.21.15.46:6839/3771802136] up:boot 2024-04-17T08:03:04.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: fsmap 12 up:standby 2024-04-17T08:03:04.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T08:03:04.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T08:03:04.723 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T08:03:04.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T08:03:04.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T08:03:04.724 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:04 smithi053 ceph-mon[36437]: from='mgr.14152 172.21.15.46:0/836234120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T08:03:05.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.247+0000 7f9324a00640 1 -- 172.21.15.46:0/3926274446 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7f931009c9c0 con 0x7f9320101da0 2024-04-17T08:03:05.246 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-17T08:03:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.252+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93040a3c20 msgr2=0x7f93040a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.252+0000 7f931fe00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93040a3c20 0x7f93040a60e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f930c0095f0 tx=0x7f930c005f90 comp rx=0 tx=0).stop 2024-04-17T08:03:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.252+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320101da0 msgr2=0x7f9320198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:05.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.252+0000 7f931fe00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320101da0 0x7f9320198000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f931000e9c0 tx=0x7f931000ee80 comp rx=0 tx=0).stop 2024-04-17T08:03:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.254+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 shutdown_connections 2024-04-17T08:03:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.254+0000 7f931fe00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f93040a3c20 0x7f93040a60e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.254+0000 7f931fe00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9320103950 0x7f932019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.254+0000 7f931fe00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9320102f90 0x7f9320198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.254+0000 7f931fe00640 1 --2- 172.21.15.46:0/3926274446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9320101da0 0x7f9320198000 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.254+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 >> 172.21.15.46:0/3926274446 conn(0x7f93200fd310 msgr2=0x7f9320106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:05.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.256+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 shutdown_connections 2024-04-17T08:03:05.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.256+0000 7f931fe00640 1 -- 172.21.15.46:0/3926274446 wait complete. 2024-04-17T08:03:05.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T08:03:05.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 -- 172.21.15.46:0/1615910176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 msgr2=0x7f35a8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 --2- 172.21.15.46:0/1615910176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 0x7f35a8101db0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3598009900 tx=0x7f359802bc30 comp rx=0 tx=0).stop 2024-04-17T08:03:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 -- 172.21.15.46:0/1615910176 shutdown_connections 2024-04-17T08:03:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 --2- 172.21.15.46:0/1615910176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 0x7f35a8101db0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 --2- 172.21.15.46:0/1615910176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8100f90 0x7f35a81013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 --2- 172.21.15.46:0/1615910176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 0x7f35a8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 -- 172.21.15.46:0/1615910176 >> 172.21.15.46:0/1615910176 conn(0x7f35a80fc250 msgr2=0x7f35a80fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:05.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 -- 172.21.15.46:0/1615910176 shutdown_connections 2024-04-17T08:03:05.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.486+0000 7f35aca00640 1 -- 172.21.15.46:0/1615910176 wait complete. 2024-04-17T08:03:05.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 Processor -- start 2024-04-17T08:03:05.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 -- start start 2024-04-17T08:03:05.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 0x7f35a819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8100f90 0x7f35a819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 0x7f35a81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810fcb0 con 0x7f35a8100f90 2024-04-17T08:03:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810fb30 con 0x7f35a8101930 2024-04-17T08:03:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.487+0000 7f35aca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810fe30 con 0x7f35a8100630 2024-04-17T08:03:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 0x7f35a819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:05.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 0x7f35a819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.46:50554/0 (socket says 172.21.15.46:50554) 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 -- 172.21.15.46:0/3952729317 learned_addr learned my addr 172.21.15.46:0/3952729317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a5e00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8100f90 0x7f35a819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a7200640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 0x7f35a81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 -- 172.21.15.46:0/3952729317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 msgr2=0x7f35a81a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 0x7f35a81a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 -- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8100f90 msgr2=0x7f35a819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8100f90 0x7f35a819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:05.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.488+0000 7f35a6800640 1 -- 172.21.15.46:0/3952729317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a81a1e80 con 0x7f35a8100630 2024-04-17T08:03:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.489+0000 7f35a6800640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 0x7f35a819a260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9f0 tx=0x7f35a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.489+0000 7f359f400640 1 -- 172.21.15.46:0/3952729317 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a000cd80 con 0x7f35a8100630 2024-04-17T08:03:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.489+0000 7f359f400640 1 -- 172.21.15.46:0/3952729317 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a0010910 con 0x7f35a8100630 2024-04-17T08:03:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.489+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a81a2170 con 0x7f35a8100630 2024-04-17T08:03:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.489+0000 7f359f400640 1 -- 172.21.15.46:0/3952729317 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a00188f0 con 0x7f35a8100630 2024-04-17T08:03:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.489+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a26c0 con 0x7f35a8100630 2024-04-17T08:03:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.490+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a810b4c0 con 0x7f35a8100630 2024-04-17T08:03:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.492+0000 7f359f400640 1 -- 172.21.15.46:0/3952729317 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f35a0010ac0 con 0x7f35a8100630 2024-04-17T08:03:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.494+0000 7f359f400640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35880a3bb0 0x7f35880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.494+0000 7f359f400640 1 -- 172.21.15.46:0/3952729317 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8825+0+0 (secure 0 0 0) 0x7f35a00d0bf0 con 0x7f35a8100630 2024-04-17T08:03:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.494+0000 7f35a5e00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35880a3bb0 0x7f35880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:05.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.495+0000 7f35a5e00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35880a3bb0 0x7f35880a6070 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f35a819b770 tx=0x7f3590009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.496+0000 7f359f400640 1 -- 172.21.15.46:0/3952729317 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f35a009ccd0 con 0x7f35a8100630 2024-04-17T08:03:05.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:05 smithi062 ceph-mon[36507]: pgmap v555: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-17T08:03:05.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:05 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3926274446' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:05.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:05 smithi046 ceph-mon[32038]: pgmap v555: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-17T08:03:05.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:05 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3926274446' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:05.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:05.708+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35a802d5d0 con 0x7f35a8100630 2024-04-17T08:03:05.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:05 smithi053 ceph-mon[36437]: pgmap v555: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-17T08:03:05.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:05 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3926274446' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:06.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.264+0000 7f359f400640 1 -- 172.21.15.46:0/3952729317 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7f35a00a0fd0 con 0x7f35a8100630 2024-04-17T08:03:06.265 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-17T08:03:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35880a3bb0 msgr2=0x7f35880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35880a3bb0 0x7f35880a6070 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f35a819b770 tx=0x7f3590009500 comp rx=0 tx=0).stop 2024-04-17T08:03:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 msgr2=0x7f35a819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 0x7f35a819a260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9f0 tx=0x7f35a000eeb0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 shutdown_connections 2024-04-17T08:03:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f35880a3bb0 0x7f35880a6070 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35a8101930 0x7f35a81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8100f90 0x7f35a819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 --2- 172.21.15.46:0/3952729317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35a8100630 0x7f35a819a260 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.270+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 >> 172.21.15.46:0/3952729317 conn(0x7f35a80fc250 msgr2=0x7f35a80fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:06.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.271+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 shutdown_connections 2024-04-17T08:03:06.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.271+0000 7f35aca00640 1 -- 172.21.15.46:0/3952729317 wait complete. 2024-04-17T08:03:06.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-17T08:03:06.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 -- 172.21.15.46:0/2198431487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2344075fa0 msgr2=0x7f23440743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 --2- 172.21.15.46:0/2198431487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2344075fa0 0x7f23440743f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2334009ca0 tx=0x7f233402c100 comp rx=0 tx=0).stop 2024-04-17T08:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 -- 172.21.15.46:0/2198431487 shutdown_connections 2024-04-17T08:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 --2- 172.21.15.46:0/2198431487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f234410e580 0x7f2344110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 --2- 172.21.15.46:0/2198431487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23440749c0 0x7f2344074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 --2- 172.21.15.46:0/2198431487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2344075fa0 0x7f23440743f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 -- 172.21.15.46:0/2198431487 >> 172.21.15.46:0/2198431487 conn(0x7f23440fd4e0 msgr2=0x7f23440ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:06.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.521+0000 7f2348e00640 1 -- 172.21.15.46:0/2198431487 shutdown_connections 2024-04-17T08:03:06.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.522+0000 7f2348e00640 1 -- 172.21.15.46:0/2198431487 wait complete. 2024-04-17T08:03:06.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.522+0000 7f2348e00640 1 Processor -- start 2024-04-17T08:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2348e00640 1 -- start start 2024-04-17T08:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2348e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23440749c0 0x7f234419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2348e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344075fa0 0x7f234419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2348e00640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234410e580 0x7f23441a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2348e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2344114200 con 0x7f2344075fa0 2024-04-17T08:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2348e00640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2344114080 con 0x7f234410e580 2024-04-17T08:03:06.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2348e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2344114380 con 0x7f23440749c0 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2341400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344075fa0 0x7f234419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2341400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344075fa0 0x7f234419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39660/0 (socket says 172.21.15.46:39660) 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2341400640 1 -- 172.21.15.46:0/3177989100 learned_addr learned my addr 172.21.15.46:0/3177989100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f2341e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23440749c0 0x7f234419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.523+0000 7f2342800640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234410e580 0x7f23441a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f2341400640 1 -- 172.21.15.46:0/3177989100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23440749c0 msgr2=0x7f234419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f2341400640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23440749c0 0x7f234419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f2341400640 1 -- 172.21.15.46:0/3177989100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234410e580 msgr2=0x7f23441a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f2341400640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234410e580 0x7f23441a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f2341400640 1 -- 172.21.15.46:0/3177989100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23441a64d0 con 0x7f2344075fa0 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f2341400640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344075fa0 0x7f234419ee40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e9f0 tx=0x7f232c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.524+0000 7f233aa00640 1 -- 172.21.15.46:0/3177989100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c00cd80 con 0x7f2344075fa0 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.525+0000 7f233aa00640 1 -- 172.21.15.46:0/3177989100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232c010910 con 0x7f2344075fa0 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.525+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23441a67c0 con 0x7f2344075fa0 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.525+0000 7f233aa00640 1 -- 172.21.15.46:0/3177989100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c0188f0 con 0x7f2344075fa0 2024-04-17T08:03:06.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.525+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23441a6d60 con 0x7f2344075fa0 2024-04-17T08:03:06.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.526+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2344104750 con 0x7f2344075fa0 2024-04-17T08:03:06.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.528+0000 7f233aa00640 1 -- 172.21.15.46:0/3177989100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f232c010ac0 con 0x7f2344075fa0 2024-04-17T08:03:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.529+0000 7f233aa00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f23280a3c30 0x7f23280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T08:03:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.529+0000 7f233aa00640 1 -- 172.21.15.46:0/3177989100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8324+0+0 (secure 0 0 0) 0x7f232c0d07c0 con 0x7f2344075fa0 2024-04-17T08:03:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.530+0000 7f2341e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f23280a3c30 0x7f23280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T08:03:06.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.530+0000 7f2341e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f23280a3c30 0x7f23280a60f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f233402c710 tx=0x7f23340323c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T08:03:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.534+0000 7f233aa00640 1 -- 172.21.15.46:0/3177989100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f232c09ca90 con 0x7f2344075fa0 2024-04-17T08:03:06.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:06 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3926274446' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T08:03:06.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:06 smithi062 ceph-mon[36507]: osdmap e95: 12 total, 12 up, 12 in 2024-04-17T08:03:06.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:06 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3952729317' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:06.553 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:06 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:06.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:06 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3926274446' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T08:03:06.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:06 smithi046 ceph-mon[32038]: osdmap e95: 12 total, 12 up, 12 in 2024-04-17T08:03:06.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:06 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3952729317' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:06.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:06 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:06.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:06 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3926274446' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T08:03:06.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:06 smithi053 ceph-mon[36437]: osdmap e95: 12 total, 12 up, 12 in 2024-04-17T08:03:06.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:06 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3952729317' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:06.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:06 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:06.764+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2344062b60 con 0x7f2344075fa0 2024-04-17T08:03:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.261+0000 7f233aa00640 1 -- 172.21.15.46:0/3177989100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) ==== 178+0+0 (secure 0 0 0) 0x7f232c0a0d90 con 0x7f2344075fa0 2024-04-17T08:03:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' removed 2024-04-17T08:03:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.268+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f23280a3c30 msgr2=0x7f23280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.268+0000 7f2348e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f23280a3c30 0x7f23280a60f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f233402c710 tx=0x7f23340323c0 comp rx=0 tx=0).stop 2024-04-17T08:03:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.268+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344075fa0 msgr2=0x7f234419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T08:03:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.268+0000 7f2348e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344075fa0 0x7f234419ee40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e9f0 tx=0x7f232c00eeb0 comp rx=0 tx=0).stop 2024-04-17T08:03:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.270+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 shutdown_connections 2024-04-17T08:03:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.270+0000 7f2348e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:6800/4172081855,v1:172.21.15.46:6801/4172081855] conn(0x7f23280a3c30 0x7f23280a60f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.270+0000 7f2348e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234410e580 0x7f23441a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.270+0000 7f2348e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344075fa0 0x7f234419ee40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.270+0000 7f2348e00640 1 --2- 172.21.15.46:0/3177989100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23440749c0 0x7f234419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T08:03:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.270+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 >> 172.21.15.46:0/3177989100 conn(0x7f23440fd4e0 msgr2=0x7f2344101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T08:03:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.270+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 shutdown_connections 2024-04-17T08:03:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-17T08:03:07.271+0000 7f2348e00640 1 -- 172.21.15.46:0/3177989100 wait complete. 2024-04-17T08:03:07.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T08:03:07.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T08:03:07.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T08:03:07.467 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-17T08:03:07.467 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T08:03:07.522 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T08:03:07.539 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T08:03:07.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:07 smithi062 ceph-mon[36507]: pgmap v557: 97 pgs: 97 active+clean; 26 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T08:03:07.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:07 smithi062 ceph-mon[36507]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T08:03:07.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:07 smithi062 ceph-mon[36507]: osdmap e96: 12 total, 12 up, 12 in 2024-04-17T08:03:07.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:07 smithi062 ceph-mon[36507]: from='client.? 172.21.15.46:0/3177989100' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:07.554 INFO:tasks.cephadm:Stopping all daemons... 2024-04-17T08:03:07.554 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-17T08:03:07.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a 2024-04-17T08:03:07.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:07 smithi046 ceph-mon[32038]: pgmap v557: 97 pgs: 97 active+clean; 26 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T08:03:07.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:07 smithi046 ceph-mon[32038]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T08:03:07.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:07 smithi046 ceph-mon[32038]: osdmap e96: 12 total, 12 up, 12 in 2024-04-17T08:03:07.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:07 smithi046 ceph-mon[32038]: from='client.? 172.21.15.46:0/3177989100' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:07.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:07 smithi053 ceph-mon[36437]: pgmap v557: 97 pgs: 97 active+clean; 26 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T08:03:07.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:07 smithi053 ceph-mon[36437]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-17T08:03:07.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:07 smithi053 ceph-mon[36437]: osdmap e96: 12 total, 12 up, 12 in 2024-04-17T08:03:07.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:07 smithi053 ceph-mon[36437]: from='client.? 172.21.15.46:0/3177989100' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-17T08:03:08.110 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:07 smithi046 systemd[1]: Stopping Ceph mon.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:08.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:08 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a[32034]: 2024-04-17T08:03:08.230+0000 7f9d85a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:08.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:08 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a[32034]: 2024-04-17T08:03:08.230+0000 7f9d85a00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-17T08:03:08.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:08 smithi046 podman[99539]: 2024-04-17 08:03:08.499054325 +0000 UTC m=+0.495019609 container died 18918c5ec8f3d10f77477ecf6e042ce5fe4f06c1f6cfa3c6e0e563988918d675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-17T08:03:09.317 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:09 smithi046 podman[99539]: 2024-04-17 08:03:09.056669184 +0000 UTC m=+1.052634477 container cleanup 18918c5ec8f3d10f77477ecf6e042ce5fe4f06c1f6cfa3c6e0e563988918d675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-17T08:03:09.317 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:09 smithi046 bash[99539]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a 2024-04-17T08:03:09.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Apr 17 08:03:09 smithi046 podman[99553]: 2024-04-17 08:03:09.316966 +0000 UTC m=+0.801729554 container remove 18918c5ec8f3d10f77477ecf6e042ce5fe4f06c1f6cfa3c6e0e563988918d675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-17T08:03:09.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.a.service' 2024-04-17T08:03:09.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:09.953 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-17T08:03:09.953 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-17T08:03:09.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.b 2024-04-17T08:03:10.275 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:10 smithi053 systemd[1]: Stopping Ceph mon.b for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:10.720 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:10 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-b[36433]: 2024-04-17T08:03:10.276+0000 7fd87ebe9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:10.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:10 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-b[36433]: 2024-04-17T08:03:10.276+0000 7fd87ebe9700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-17T08:03:10.721 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:10 smithi053 podman[70503]: 2024-04-17 08:03:10.440023047 +0000 UTC m=+0.278064576 container died 4941649dc87c93d78e655b8cb53380f5905491b5bb33306b56368250f8ba69c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-17T08:03:11.119 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:10 smithi053 podman[70503]: 2024-04-17 08:03:10.82765892 +0000 UTC m=+0.665700448 container cleanup 4941649dc87c93d78e655b8cb53380f5905491b5bb33306b56368250f8ba69c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-17T08:03:11.119 INFO:journalctl@ceph.mon.b.smithi053.stdout:Apr 17 08:03:10 smithi053 bash[70503]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-b 2024-04-17T08:03:11.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.b.service' 2024-04-17T08:03:11.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:11.301 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-17T08:03:11.301 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-17T08:03:11.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.c 2024-04-17T08:03:11.628 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:11 smithi062 systemd[1]: Stopping Ceph mon.c for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:11.944 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-c[36501]: 2024-04-17T08:03:11.629+0000 7fa823297700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:11.945 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-c[36501]: 2024-04-17T08:03:11.629+0000 7fa823297700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-17T08:03:11.945 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:11 smithi062 podman[71290]: 2024-04-17 08:03:11.731730435 +0000 UTC m=+0.226401758 container died f7dc232f5d831eba960eb9cf2489ababf5dfec0ccdf33473f14609cd9443bef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-17T08:03:12.213 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:12 smithi062 podman[71290]: 2024-04-17 08:03:12.12987631 +0000 UTC m=+0.624547618 container cleanup f7dc232f5d831eba960eb9cf2489ababf5dfec0ccdf33473f14609cd9443bef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-17T08:03:12.214 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:12 smithi062 bash[71290]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-c 2024-04-17T08:03:12.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Apr 17 08:03:12 smithi062 podman[71305]: 2024-04-17 08:03:12.390031866 +0000 UTC m=+0.653855235 container remove f7dc232f5d831eba960eb9cf2489ababf5dfec0ccdf33473f14609cd9443bef3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:03:12.577 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mon.c.service' 2024-04-17T08:03:12.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:12.637 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-17T08:03:12.637 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-17T08:03:12.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.x 2024-04-17T08:03:13.109 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 08:03:12 smithi046 systemd[1]: Stopping Ceph mgr.x for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:13.801 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 08:03:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T08:03:13.403+0000 7fc2c0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:13.801 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 08:03:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x[32267]: 2024-04-17T08:03:13.403+0000 7fc2c0a00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-17T08:03:13.801 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 08:03:13 smithi046 podman[99658]: 2024-04-17 08:03:13.497150388 +0000 UTC m=+0.407442632 container died 4e3930768b4a74e4144d593eb544ac006578513fae0a5086a3a75968442ede14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-17T08:03:14.055 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 08:03:14 smithi046 podman[99658]: 2024-04-17 08:03:14.045641338 +0000 UTC m=+0.955933565 container cleanup 4e3930768b4a74e4144d593eb544ac006578513fae0a5086a3a75968442ede14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-17T08:03:14.055 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 08:03:14 smithi046 bash[99658]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x 2024-04-17T08:03:14.859 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Apr 17 08:03:14 smithi046 podman[99671]: 2024-04-17 08:03:14.398536733 +0000 UTC m=+0.883880953 container remove 4e3930768b4a74e4144d593eb544ac006578513fae0a5086a3a75968442ede14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-x, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:03:14.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.x.service' 2024-04-17T08:03:15.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:15.083 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-17T08:03:15.083 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-17T08:03:15.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.y 2024-04-17T08:03:15.451 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 08:03:15 smithi053 systemd[1]: Stopping Ceph mgr.y for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:15.720 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 08:03:15 smithi053 podman[70628]: 2024-04-17 08:03:15.453251382 +0000 UTC m=+0.168557149 container died 10ba0a837158b07ce5c56603d68373c20d62554a05dcc78505a3a3e9b22440f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-17T08:03:16.083 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 08:03:15 smithi053 podman[70628]: 2024-04-17 08:03:15.850130604 +0000 UTC m=+0.565436371 container cleanup 10ba0a837158b07ce5c56603d68373c20d62554a05dcc78505a3a3e9b22440f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-17T08:03:16.083 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Apr 17 08:03:15 smithi053 bash[70628]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-y 2024-04-17T08:03:16.216 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.y.service' 2024-04-17T08:03:16.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:16.258 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-17T08:03:16.258 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-17T08:03:16.258 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.z 2024-04-17T08:03:16.553 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 08:03:16 smithi062 systemd[1]: Stopping Ceph mgr.z for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:17.049 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 08:03:16 smithi062 podman[71425]: 2024-04-17 08:03:16.690473753 +0000 UTC m=+0.231664283 container died 35a0481921de92e8361416940b0ae971cbb428422a32d56cd270c3b829720fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-17T08:03:17.302 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 08:03:17 smithi062 podman[71425]: 2024-04-17 08:03:17.228068811 +0000 UTC m=+0.769259339 container cleanup 35a0481921de92e8361416940b0ae971cbb428422a32d56cd270c3b829720fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:03:17.302 INFO:journalctl@ceph.mgr.z.smithi062.stdout:Apr 17 08:03:17 smithi062 bash[71425]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mgr-z 2024-04-17T08:03:17.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mgr.z.service' 2024-04-17T08:03:17.699 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:17.700 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-17T08:03:17.700 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-17T08:03:17.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.0 2024-04-17T08:03:18.109 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:17 smithi046 systemd[1]: Stopping Ceph osd.0 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:18.609 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0[46987]: 2024-04-17T08:03:18.303+0000 7fb6c8a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:18.610 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0[46987]: 2024-04-17T08:03:18.303+0000 7fb6c8a00700 -1 osd.0 97 *** Got signal Terminated *** 2024-04-17T08:03:18.610 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0[46987]: 2024-04-17T08:03:18.303+0000 7fb6c8a00700 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:03:24.359 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:23 smithi046 podman[99778]: 2024-04-17 08:03:23.891064141 +0000 UTC m=+5.762481292 container died 38ac4eb509b91a17a78834c1a681b0f9f078150278264b7feb7bc1bb946c0610 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-17T08:03:24.839 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:24 smithi046 podman[99778]: 2024-04-17 08:03:24.579808616 +0000 UTC m=+6.451225777 container cleanup 38ac4eb509b91a17a78834c1a681b0f9f078150278264b7feb7bc1bb946c0610 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:03:24.839 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:24 smithi046 bash[99778]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0 2024-04-17T08:03:25.109 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:24 smithi046 podman[99799]: 2024-04-17 08:03:24.839746223 +0000 UTC m=+0.937236414 container remove 38ac4eb509b91a17a78834c1a681b0f9f078150278264b7feb7bc1bb946c0610 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T08:03:26.110 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:25 smithi046 podman[99888]: 2024-04-17 08:03:25.608609745 +0000 UTC m=+0.035286757 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:03:26.110 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:25 smithi046 podman[99888]: 2024-04-17 08:03:25.732577489 +0000 UTC m=+0.159254518 container create 22c9d18ec5380db3e7e858a074d56dc5c401a232f5bdb17fd6a20d104c17fa04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-17T08:03:26.609 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:26 smithi046 podman[99888]: 2024-04-17 08:03:26.203759862 +0000 UTC m=+0.630436931 container init 22c9d18ec5380db3e7e858a074d56dc5c401a232f5bdb17fd6a20d104c17fa04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:03:26.609 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:26 smithi046 podman[99888]: 2024-04-17 08:03:26.215287921 +0000 UTC m=+0.641964953 container start 22c9d18ec5380db3e7e858a074d56dc5c401a232f5bdb17fd6a20d104c17fa04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-17T08:03:26.610 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:26 smithi046 podman[99888]: 2024-04-17 08:03:26.310823684 +0000 UTC m=+0.737500763 container attach 22c9d18ec5380db3e7e858a074d56dc5c401a232f5bdb17fd6a20d104c17fa04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:03:27.009 INFO:journalctl@ceph.osd.0.smithi046.stdout:Apr 17 08:03:26 smithi046 podman[99888]: 2024-04-17 08:03:26.696854901 +0000 UTC m=+1.123531933 container died 22c9d18ec5380db3e7e858a074d56dc5c401a232f5bdb17fd6a20d104c17fa04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-17T08:03:27.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.0.service' 2024-04-17T08:03:27.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:27.635 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-17T08:03:27.635 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-17T08:03:27.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.1 2024-04-17T08:03:27.945 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:27 smithi053 systemd[1]: Stopping Ceph osd.1 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:28.221 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1[41028]: 2024-04-17T08:03:27.946+0000 7f0b14c98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:28.221 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1[41028]: 2024-04-17T08:03:27.946+0000 7f0b14c98700 -1 osd.1 97 *** Got signal Terminated *** 2024-04-17T08:03:28.221 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1[41028]: 2024-04-17T08:03:27.946+0000 7f0b14c98700 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:03:33.721 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:33 smithi053 podman[70763]: 2024-04-17 08:03:33.460684207 +0000 UTC m=+5.621118825 container died 7b3e8fd1269b4f796185fcfd200d58a32388aac8fb7c8c49e612b05edf7735d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-17T08:03:34.221 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:33 smithi053 podman[70763]: 2024-04-17 08:03:33.796925533 +0000 UTC m=+5.957360152 container cleanup 7b3e8fd1269b4f796185fcfd200d58a32388aac8fb7c8c49e612b05edf7735d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-17T08:03:34.221 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:33 smithi053 bash[70763]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1 2024-04-17T08:03:34.221 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:33 smithi053 podman[70780]: 2024-04-17 08:03:33.965159122 +0000 UTC m=+0.503701016 container remove 7b3e8fd1269b4f796185fcfd200d58a32388aac8fb7c8c49e612b05edf7735d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T08:03:34.642 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:34 smithi053 podman[70871]: 2024-04-17 08:03:34.270855087 +0000 UTC m=+0.018320054 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:03:34.642 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:34 smithi053 podman[70871]: 2024-04-17 08:03:34.374793303 +0000 UTC m=+0.122258309 container create 6f086899cf266e1378ef50f5cf3fefddc3fa4e352ca5e1e9715eb49c8c9ae2ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-17T08:03:34.954 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:34 smithi053 podman[70871]: 2024-04-17 08:03:34.644044859 +0000 UTC m=+0.391509827 container init 6f086899cf266e1378ef50f5cf3fefddc3fa4e352ca5e1e9715eb49c8c9ae2ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:03:34.954 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:34 smithi053 podman[70871]: 2024-04-17 08:03:34.647451426 +0000 UTC m=+0.394916393 container start 6f086899cf266e1378ef50f5cf3fefddc3fa4e352ca5e1e9715eb49c8c9ae2ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-17T08:03:34.954 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:34 smithi053 podman[70871]: 2024-04-17 08:03:34.761033973 +0000 UTC m=+0.508498960 container attach 6f086899cf266e1378ef50f5cf3fefddc3fa4e352ca5e1e9715eb49c8c9ae2ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-17T08:03:35.212 INFO:journalctl@ceph.osd.1.smithi053.stdout:Apr 17 08:03:34 smithi053 podman[70871]: 2024-04-17 08:03:34.961687628 +0000 UTC m=+0.709152596 container died 6f086899cf266e1378ef50f5cf3fefddc3fa4e352ca5e1e9715eb49c8c9ae2ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-17T08:03:35.482 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.1.service' 2024-04-17T08:03:35.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:35.532 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-17T08:03:35.532 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-17T08:03:35.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.2 2024-04-17T08:03:35.857 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:35 smithi062 systemd[1]: Stopping Ceph osd.2 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:36.302 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2[41678]: 2024-04-17T08:03:35.858+0000 7f45332f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:36.302 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2[41678]: 2024-04-17T08:03:35.858+0000 7f45332f4700 -1 osd.2 97 *** Got signal Terminated *** 2024-04-17T08:03:36.302 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2[41678]: 2024-04-17T08:03:35.858+0000 7f45332f4700 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:03:41.552 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:41 smithi062 podman[71552]: 2024-04-17 08:03:41.26941595 +0000 UTC m=+5.535037519 container died 1122ab56a221cccd74d321109b3d9b670fa4e73bbdaac1a0e6ae7db6bc28df4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-17T08:03:41.928 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:41 smithi062 podman[71552]: 2024-04-17 08:03:41.68586246 +0000 UTC m=+5.951484032 container cleanup 1122ab56a221cccd74d321109b3d9b670fa4e73bbdaac1a0e6ae7db6bc28df4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-17T08:03:41.928 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:41 smithi062 bash[71552]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2 2024-04-17T08:03:42.256 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:41 smithi062 podman[71570]: 2024-04-17 08:03:41.929977535 +0000 UTC m=+0.653557291 container remove 1122ab56a221cccd74d321109b3d9b670fa4e73bbdaac1a0e6ae7db6bc28df4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-17T08:03:42.515 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:42 smithi062 podman[71659]: 2024-04-17 08:03:42.161488541 +0000 UTC m=+0.018659223 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:03:42.515 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:42 smithi062 podman[71659]: 2024-04-17 08:03:42.28957966 +0000 UTC m=+0.146750365 container create 00f50e8f713922d93d3a15137060bd66caa6dcb155faca91a7ea8933a5b0991e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:03:42.516 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:42 smithi062 podman[71659]: 2024-04-17 08:03:42.517262443 +0000 UTC m=+0.374433117 container init 00f50e8f713922d93d3a15137060bd66caa6dcb155faca91a7ea8933a5b0991e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-17T08:03:42.802 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:42 smithi062 podman[71659]: 2024-04-17 08:03:42.52073604 +0000 UTC m=+0.377906695 container start 00f50e8f713922d93d3a15137060bd66caa6dcb155faca91a7ea8933a5b0991e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-17T08:03:42.802 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:42 smithi062 podman[71659]: 2024-04-17 08:03:42.625697137 +0000 UTC m=+0.482867818 container attach 00f50e8f713922d93d3a15137060bd66caa6dcb155faca91a7ea8933a5b0991e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-17T08:03:43.185 INFO:journalctl@ceph.osd.2.smithi062.stdout:Apr 17 08:03:42 smithi062 podman[71659]: 2024-04-17 08:03:42.823066469 +0000 UTC m=+0.680237134 container died 00f50e8f713922d93d3a15137060bd66caa6dcb155faca91a7ea8933a5b0991e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-17T08:03:43.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.2.service' 2024-04-17T08:03:43.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:43.583 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-17T08:03:43.584 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-17T08:03:43.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.3 2024-04-17T08:03:44.109 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:43 smithi046 systemd[1]: Stopping Ceph osd.3 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:44.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:44 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T08:03:44.258+0000 7fac83c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:44.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:44 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T08:03:44.258+0000 7fac83c00700 -1 osd.3 97 *** Got signal Terminated *** 2024-04-17T08:03:44.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:44 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T08:03:44.258+0000 7fac83c00700 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:03:45.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:45.263+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:46.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:45 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:45.763+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:46.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:45.822+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:46.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:46.233+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:47.052 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:46.799+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:47.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:46 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:46.759+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:47.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:47 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T08:03:47.144+0000 7fac7a400700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:23.028472+0000 front 2024-04-17T08:03:23.027934+0000 (oldest deadline 2024-04-17T08:03:47.128036+0000) 2024-04-17T08:03:47.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:47.261+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:48.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:47 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:47.711+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:48.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:47.843+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:48.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:48.219+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:48.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:48 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T08:03:48.172+0000 7fac7a400700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:23.028472+0000 front 2024-04-17T08:03:23.027934+0000 (oldest deadline 2024-04-17T08:03:47.128036+0000) 2024-04-17T08:03:49.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:48 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:48.696+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:49.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:48.832+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:49.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:49.192+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:49.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:49 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3[53538]: 2024-04-17T08:03:49.169+0000 7fac7a400700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:23.028472+0000 front 2024-04-17T08:03:23.027934+0000 (oldest deadline 2024-04-17T08:03:47.128036+0000) 2024-04-17T08:03:50.111 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:49 smithi046 podman[100006]: 2024-04-17 08:03:49.800616707 +0000 UTC m=+5.728845362 container died 5c754b60b7f94a18b954e1edbaf3220fbe12f5b8ba011843baf32b3aeca81107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-17T08:03:50.111 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:49 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:49.729+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:50.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:49.839+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:50.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:50 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:50.170+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:50.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:50 smithi046 podman[100006]: 2024-04-17 08:03:50.351931058 +0000 UTC m=+6.280159720 container cleanup 5c754b60b7f94a18b954e1edbaf3220fbe12f5b8ba011843baf32b3aeca81107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-17T08:03:50.610 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:50 smithi046 bash[100006]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3 2024-04-17T08:03:51.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:50 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:50.710+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:51.109 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:50 smithi046 podman[100027]: 2024-04-17 08:03:50.653595688 +0000 UTC m=+0.832417260 container remove 5c754b60b7f94a18b954e1edbaf3220fbe12f5b8ba011843baf32b3aeca81107 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-17T08:03:51.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:50.869+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:51.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:51 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:51.215+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:51.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:51 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:51.675+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:51.860 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:51 smithi046 podman[100117]: 2024-04-17 08:03:51.442568585 +0000 UTC m=+0.038333666 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:03:51.860 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:51 smithi046 podman[100117]: 2024-04-17 08:03:51.614965128 +0000 UTC m=+0.210730242 container create 5bb3022c78f0f2425ceb06827e1e830b1ccb14f0a8738dec625604b0ec3ce8c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS) 2024-04-17T08:03:52.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:51.891+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:52.337 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:51 smithi046 podman[100117]: 2024-04-17 08:03:51.868957488 +0000 UTC m=+0.464722608 container init 5bb3022c78f0f2425ceb06827e1e830b1ccb14f0a8738dec625604b0ec3ce8c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-17T08:03:52.338 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:51 smithi046 podman[100117]: 2024-04-17 08:03:51.880753874 +0000 UTC m=+0.476519048 container start 5bb3022c78f0f2425ceb06827e1e830b1ccb14f0a8738dec625604b0ec3ce8c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T08:03:52.338 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:51 smithi046 podman[100117]: 2024-04-17 08:03:51.959144056 +0000 UTC m=+0.554909210 container attach 5bb3022c78f0f2425ceb06827e1e830b1ccb14f0a8738dec625604b0ec3ce8c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T08:03:52.609 INFO:journalctl@ceph.osd.3.smithi046.stdout:Apr 17 08:03:52 smithi046 podman[100117]: 2024-04-17 08:03:52.338924397 +0000 UTC m=+0.934689518 container died 5bb3022c78f0f2425ceb06827e1e830b1ccb14f0a8738dec625604b0ec3ce8c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T08:03:52.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:52 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:52.264+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:53.046 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:52 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:52.637+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:53.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.3.service' 2024-04-17T08:03:53.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:03:53.285 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-17T08:03:53.285 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-17T08:03:53.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.4 2024-04-17T08:03:53.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:52.849+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:53.600 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:53 smithi053 systemd[1]: Stopping Ceph osd.4 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:03:53.600 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:53 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:53.288+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:53.970 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:53 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4[47598]: 2024-04-17T08:03:53.600+0000 7fe7742bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:03:53.971 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:53 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4[47598]: 2024-04-17T08:03:53.600+0000 7fe7742bc700 -1 osd.4 97 *** Got signal Terminated *** 2024-04-17T08:03:53.971 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:53 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4[47598]: 2024-04-17T08:03:53.600+0000 7fe7742bc700 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:03:53.999 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:53 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:53.680+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:54.260 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:53.832+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:54.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:54 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:54.301+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:55.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:54 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:54.692+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:55.220 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:03:54 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:03:54.915+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:03:55.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:54.859+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:55.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:54.859+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:03:55.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:55 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:55.313+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:55.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:55 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:55.313+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:03:55.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:03:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:03:55.348+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:03:56.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:55 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:55.677+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:56.220 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:03:55 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:03:55.894+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:03:56.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:55.888+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:56.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:55.888+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:03:56.719 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:03:56 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:03:56.388+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:03:56.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:56 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:56.360+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:56.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:56 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:56.360+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:03:56.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:03:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:03:56.303+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:03:57.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:56 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:56.719+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:57.220 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:03:56 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:03:56.866+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:03:57.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:03:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:03:56.990+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:03:57.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:56.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:57.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:56.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:03:57.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:03:57 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:03:57.344+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:03:57.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:57 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:57.359+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:57.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:57 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:57.359+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:03:57.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:03:57 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:03:57.348+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:03:58.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:57 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:57.680+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:58.220 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:57 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4[47598]: 2024-04-17T08:03:57.881+0000 7fe76c8a8700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.961393+0000 front 2024-04-17T08:03:31.961423+0000 (oldest deadline 2024-04-17T08:03:57.861555+0000) 2024-04-17T08:03:58.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:03:57 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:03:57.883+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:03:58.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:03:57 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:03:57.949+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:03:58.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:57 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:57.867+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:58.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:57 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:57.867+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:03:58.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:03:58 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:03:58.299+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:03:58.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:58 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:58.388+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:58.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:58 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:58.388+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:03:58.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:03:58 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:03:58.315+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:03:59.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:58 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:58.681+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:03:59.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:58 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:58.681+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:03:59.221 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:59 smithi053 podman[70983]: 2024-04-17 08:03:59.050712206 +0000 UTC m=+5.562249945 container died 0c47e08fe141bd0dd66f258d0d12c3fe0eb56bb97f668fe5f4deb8ee05a4822e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-17T08:03:59.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:03:58 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:03:58.874+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:03:59.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:03:58 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:03:58.963+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:03:59.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:58 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:58.865+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:03:59.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:58 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:58.865+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:03:59.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:03:59 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:03:59.261+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:03:59.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:59 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:59.408+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:03:59.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:03:59 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:03:59.408+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:03:59.721 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:59 smithi053 podman[70983]: 2024-04-17 08:03:59.435001425 +0000 UTC m=+5.946539146 container cleanup 0c47e08fe141bd0dd66f258d0d12c3fe0eb56bb97f668fe5f4deb8ee05a4822e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2) 2024-04-17T08:03:59.721 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:59 smithi053 bash[70983]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4 2024-04-17T08:03:59.721 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:59 smithi053 podman[71000]: 2024-04-17 08:03:59.577887224 +0000 UTC m=+0.530245514 container remove 0c47e08fe141bd0dd66f258d0d12c3fe0eb56bb97f668fe5f4deb8ee05a4822e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-17T08:03:59.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:03:59 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:03:59.332+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:00.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:59 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:59.644+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:00.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:03:59 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:03:59.644+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:00.213 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:59 smithi053 podman[71091]: 2024-04-17 08:03:59.978908136 +0000 UTC m=+0.107838613 container create 7bcbe37e4e3eaed0678892dcec8e563fca12196a007486bbe4d2981932a1ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-17T08:04:00.214 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:03:59 smithi053 podman[71091]: 2024-04-17 08:03:59.88887879 +0000 UTC m=+0.017809287 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:00.214 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:03:59 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:03:59.890+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:00.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:03:59 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:03:59.931+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:04:00.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:59 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:59.906+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:00.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:03:59 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:03:59.906+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:00.471 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:04:00 smithi053 podman[71091]: 2024-04-17 08:04:00.215150208 +0000 UTC m=+0.344080702 container init 7bcbe37e4e3eaed0678892dcec8e563fca12196a007486bbe4d2981932a1ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-17T08:04:00.471 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:04:00 smithi053 podman[71091]: 2024-04-17 08:04:00.21846503 +0000 UTC m=+0.347395518 container start 7bcbe37e4e3eaed0678892dcec8e563fca12196a007486bbe4d2981932a1ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-17T08:04:00.471 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:04:00 smithi053 podman[71091]: 2024-04-17 08:04:00.306764522 +0000 UTC m=+0.435695017 container attach 7bcbe37e4e3eaed0678892dcec8e563fca12196a007486bbe4d2981932a1ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-17T08:04:00.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:00 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:00.396+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:00.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:00 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:00.396+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:00.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:00 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:00.283+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:00.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:00 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:00.338+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:00.808 INFO:journalctl@ceph.osd.4.smithi053.stdout:Apr 17 08:04:00 smithi053 podman[71091]: 2024-04-17 08:04:00.525005428 +0000 UTC m=+0.653935933 container died 7bcbe37e4e3eaed0678892dcec8e563fca12196a007486bbe4d2981932a1ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:04:01.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:00 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:00.647+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:01.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:00 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:00.647+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:01.110 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:00 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:00.922+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:01.126 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.4.service' 2024-04-17T08:04:01.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:01.175 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-17T08:04:01.175 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-17T08:04:01.175 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.5 2024-04-17T08:04:01.209 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:00 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:00.957+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:04:01.210 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:00 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:00.885+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:01.210 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:00 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:00.885+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:01.498 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:01 smithi062 systemd[1]: Stopping Ceph osd.5 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:04:01.499 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:01.347+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:01.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:01 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:01.287+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:01.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:01 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:01.407+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:01.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:01 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:01.407+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:01.802 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:01.498+0000 7fd09d70f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:04:01.803 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:01.498+0000 7fd09d70f700 -1 osd.5 97 *** Got signal Terminated *** 2024-04-17T08:04:01.803 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:01.498+0000 7fd09d70f700 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:04:02.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:01 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:01.619+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:02.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:01 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:01.619+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:02.220 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:01 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:01.923+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:02.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:01.975+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:04:02.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:01.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:02.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:01.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:02.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:01 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:01.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:02.592 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:02 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:02.303+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:02.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:02.376+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:02.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:02.376+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:02.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:02.392+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:02.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:02.392+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:02.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:02 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:02.592+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:02.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:02 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:02.592+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:03.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:02 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:02.939+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:03.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:02.950+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:04:03.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:02.960+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:03.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:02.960+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:03.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:02 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:02.960+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:03.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:03 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:03.321+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:03.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:03 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:03.322+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:03.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:03 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:03.566+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:03.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:03 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:03.566+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:03.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:03 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:03.566+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:03.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:03 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:03.381+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:03.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:03 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:03.381+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:03.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:03 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:03.395+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:03.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:03 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:03.395+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:04.220 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:03 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:03.964+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:04.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:03 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:03.964+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:04.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:03 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:03.991+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:04:04.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:03 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:03.991+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.764587+0000 front 2024-04-17T08:03:40.764625+0000 (oldest deadline 2024-04-17T08:04:03.664720+0000) 2024-04-17T08:04:04.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:03 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:03.944+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:04.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:03 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:03.944+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:04.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:03 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:03.944+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:04.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:04 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:04.320+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:04.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:04 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:04.320+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:04.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:04 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:04.552+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:04.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:04 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:04.552+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:04.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:04 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:04.552+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:04.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:04.410+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:04.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:04.410+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:04.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:04.410+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:04.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:04 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:04.362+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:04.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:04 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:04.362+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:05.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:05.013+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:04:05.303 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:05.013+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.764587+0000 front 2024-04-17T08:03:40.764625+0000 (oldest deadline 2024-04-17T08:04:03.664720+0000) 2024-04-17T08:04:05.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:04 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:04.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:05.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:04 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:04.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:05.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:04 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:04.912+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:05.392 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:04.985+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:05.392 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:04 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:04.986+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:05.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:05 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:05.278+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:05.610 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:05 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:05.278+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:05.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:05 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:05.515+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:05.611 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:05 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:05.515+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:05.611 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:05 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:05.515+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:05.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:05 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:05.393+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:05.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:05 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:05.393+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:05.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:05 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:05.393+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:05.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:05.396+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:05.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:05.396+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:06.302 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:05.987+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.563499+0000 front 2024-04-17T08:03:32.563467+0000 (oldest deadline 2024-04-17T08:03:56.063609+0000) 2024-04-17T08:04:06.303 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5[48384]: 2024-04-17T08:04:05.987+0000 7fd095cfb700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.764587+0000 front 2024-04-17T08:03:40.764625+0000 (oldest deadline 2024-04-17T08:04:03.664720+0000) 2024-04-17T08:04:06.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:05.896+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:06.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:05.897+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:06.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:05 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:05.897+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:06.353 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:06 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:06.034+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:06.353 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:06 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:06.034+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:06.544 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:06 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:06.293+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:06.544 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:06 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:06.293+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:06.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:06 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:06.354+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:06.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:06 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:06.354+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:06.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:06 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:06.354+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:06.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:06 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:06.411+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:06.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:06 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:06.411+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:06.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:06 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:06.544+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:06.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:06 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:06.544+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:06.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:06 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:06.544+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:07.259 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:06 smithi062 podman[71777]: 2024-04-17 08:04:06.99144189 +0000 UTC m=+5.612736240 container died 05d95c3be5f6fda88a73e915dd468ae097184f6e0d1ab8d2c94107c359728de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:07.259 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:06 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:06.885+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:07.259 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:06 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:06.885+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:07.259 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:06 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:06.885+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:07.336 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:07 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:07.053+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:07.336 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:07 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:07.053+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:07.552 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:07 smithi062 podman[71777]: 2024-04-17 08:04:07.368674291 +0000 UTC m=+5.989968648 container cleanup 05d95c3be5f6fda88a73e915dd468ae097184f6e0d1ab8d2c94107c359728de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:04:07.553 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:07 smithi062 bash[71777]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5 2024-04-17T08:04:07.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:07 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:07.441+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:07.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:07 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:07.441+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:07.594 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:07 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:07.343+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:07.595 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:07 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:07.343+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:07.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:07 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:07.336+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:07.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:07 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:07.336+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:07.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:07 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:07.336+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:07.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:07 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:07.595+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:07.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:07 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:07.595+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:07.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:07 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:07.595+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:07.922 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:07 smithi062 podman[71795]: 2024-04-17 08:04:07.619887696 +0000 UTC m=+0.623658834 container remove 05d95c3be5f6fda88a73e915dd468ae097184f6e0d1ab8d2c94107c359728de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:08.207 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:07 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:07.923+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:08.207 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:07 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:07.923+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:08.207 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:07 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:07.923+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:08.208 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:07 smithi062 podman[71885]: 2024-04-17 08:04:07.846772285 +0000 UTC m=+0.018169278 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:08.208 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:07 smithi062 podman[71885]: 2024-04-17 08:04:07.962655916 +0000 UTC m=+0.134052909 container create fbd05ef94d2d84192a03806d49fea40fa81bcf605327b567ade00691cca7a7da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:08.208 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:08 smithi062 podman[71885]: 2024-04-17 08:04:08.208184963 +0000 UTC m=+0.379581946 container init fbd05ef94d2d84192a03806d49fea40fa81bcf605327b567ade00691cca7a7da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-17T08:04:08.350 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:08 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:08.051+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:08.351 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:08 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:08.051+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:08.469 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:08 smithi062 podman[71885]: 2024-04-17 08:04:08.211947571 +0000 UTC m=+0.383344546 container start fbd05ef94d2d84192a03806d49fea40fa81bcf605327b567ade00691cca7a7da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-17T08:04:08.469 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:08 smithi062 podman[71885]: 2024-04-17 08:04:08.307417022 +0000 UTC m=+0.478814015 container attach fbd05ef94d2d84192a03806d49fea40fa81bcf605327b567ade00691cca7a7da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS) 2024-04-17T08:04:08.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:08 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:08.383+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:08.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:08 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:08.383+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:08.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:08 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:08.351+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:08.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:08 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:08.351+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:08.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:08 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:08.351+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:08.802 INFO:journalctl@ceph.osd.5.smithi062.stdout:Apr 17 08:04:08 smithi062 podman[71885]: 2024-04-17 08:04:08.511807561 +0000 UTC m=+0.683204526 container died fbd05ef94d2d84192a03806d49fea40fa81bcf605327b567ade00691cca7a7da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:08.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:08 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:08.469+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:08.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:08 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:08.469+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:09.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:08 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:08.645+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:09.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:08 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:08.645+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:09.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:08 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:08.645+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:09.127 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:08 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:08.922+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:09.127 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:08 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:08.922+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:09.128 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:08 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:08.922+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:09.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.5.service' 2024-04-17T08:04:09.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:09.239 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-17T08:04:09.240 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-17T08:04:09.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.6 2024-04-17T08:04:09.372 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:09 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:09.072+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:09.372 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:09 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:09.072+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:09.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:09.364+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:09.610 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:09.364+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:09.610 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:09 smithi046 systemd[1]: Stopping Ceph osd.6 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:04:09.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:09 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:09.373+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:09.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:09 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:09.373+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:09.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:09 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:09.373+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:09.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:09 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:09.509+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:09.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:09 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:09.509+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:10.109 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:09.873+0000 7f72a5c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:04:10.110 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:09.873+0000 7f72a5c00700 -1 osd.6 97 *** Got signal Terminated *** 2024-04-17T08:04:10.110 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:09.873+0000 7f72a5c00700 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:04:10.110 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:09.635+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:10.110 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:09.635+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:10.110 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:09 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:09.635+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:10.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:09 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:09.894+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:10.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:09 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:09.894+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:10.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:09 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:09.894+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:10.471 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:10 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:10.120+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:10.471 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:10 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:10.120+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:10.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:10 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:10.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:10.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:10 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:10.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:10.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:10 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:10.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:10.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:10 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:10.536+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:10.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:10 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:10.536+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:10.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:10 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:10.536+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:10.859 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:10 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:10.410+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:10.859 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:10 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:10.410+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:10.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:10 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:10.621+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:10.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:10 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:10.621+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:10.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:10 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:10.621+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:11.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:10 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:10.879+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:11.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:10 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:10.879+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:11.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:10 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:10.879+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:11.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:11 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:11.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:11.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:11 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:11.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:11.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:11 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:11.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:11.471 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:11 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:11.120+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:11.471 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:11 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:11.120+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:11.472 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:11 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:11.120+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:11.859 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:11 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:11.368+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:11.859 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:11 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:11.368+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:11.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:11 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:11.575+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:11.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:11 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:11.575+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:11.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:11 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:11.575+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:11.888 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:11.576+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:11.888 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:11.576+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:11.888 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:11.576+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:12.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:11.889+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:12.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:11.889+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:12.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:11 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:11.889+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:12.347 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:12 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:12.079+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:12.347 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:12 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:12.079+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:12.347 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:12 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:12.079+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:12.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:12.340+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:12.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:12.340+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:12.609 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:12.557+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:12.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:12.557+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:12.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:12.557+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:12.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:12 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:12.557+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:12.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:12 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:12.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:12.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:12 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:12.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:12.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:12 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:12.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:12.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:12 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:12.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:12.919 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:12 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:12.591+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:12.919 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:12 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:12.591+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:12.919 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:12 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:12.591+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:13.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:12 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:12.919+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:13.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:12 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:12.919+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:13.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:12 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:12.919+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:13.339 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:13 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:13.055+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:13.339 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:13 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:13.055+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:13.339 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:13 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:13.055+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:13.568 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:13.299+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:13.569 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:13.299+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:13.569 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:13.299+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:49.009084+0000 front 2024-04-17T08:03:49.009271+0000 (oldest deadline 2024-04-17T08:04:12.509013+0000) 2024-04-17T08:04:13.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:13 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:13.340+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:13.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:13 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:13.340+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:13.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:13 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:13.340+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:13.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:13 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:13.340+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:13.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:13.569+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:13.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:13.569+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:13.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:13.569+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:13.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:13 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:13.569+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:13.966 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:13 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:13.585+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:13.967 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:13 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:13.585+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:13.967 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:13 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:13.585+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:14.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:13 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:13.967+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:14.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:13 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:13.967+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:14.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:13 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:13.967+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:14.314 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:14.024+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:14.314 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:14.024+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:14.314 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:14.024+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:14.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:14.265+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.201976+0000 front 2024-04-17T08:03:32.202061+0000 (oldest deadline 2024-04-17T08:03:55.702147+0000) 2024-04-17T08:04:14.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:14.265+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.706318+0000 front 2024-04-17T08:03:39.705866+0000 (oldest deadline 2024-04-17T08:04:02.605821+0000) 2024-04-17T08:04:14.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6[60279]: 2024-04-17T08:04:14.265+0000 7f72a1400700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:49.009084+0000 front 2024-04-17T08:03:49.009271+0000 (oldest deadline 2024-04-17T08:04:12.509013+0000) 2024-04-17T08:04:14.722 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:14.314+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:14.722 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:14.314+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:14.723 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:14.314+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:14.723 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:14.314+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:14.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:14 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:14.539+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:14.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:14 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:14.539+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:14.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:14 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:14.539+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:14.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:14.580+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:14.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:14.580+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:14.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:14.580+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:14.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:14 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:14.580+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:15.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:14 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:14.939+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:15.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:14 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:14.939+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:15.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:14 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:14.939+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:15.317 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:14.978+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:15.317 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:14.978+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:15.317 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:14 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:14.978+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:15.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:15 smithi046 podman[100235]: 2024-04-17 08:04:15.317397266 +0000 UTC m=+5.582669771 container died a9095f342142fe45ac0671672402dbc7cf8c56b0e065042530af0d05204b8d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-17T08:04:15.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:15 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:15.545+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:15.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:15 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:15.545+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:15.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:15 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:15.545+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:15.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:15 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:15.545+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:15.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:15 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:15.318+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:15.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:15 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:15.318+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:15.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:15 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:15.318+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:15.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:15 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:15.318+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:15.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:15 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:15.531+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:15.894 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:15 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:15.531+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:15.895 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:15 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:15.531+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:16.109 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:15 smithi046 podman[100235]: 2024-04-17 08:04:15.774220322 +0000 UTC m=+6.039492862 container cleanup a9095f342142fe45ac0671672402dbc7cf8c56b0e065042530af0d05204b8d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-17T08:04:16.109 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:15 smithi046 bash[100235]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6 2024-04-17T08:04:16.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:15 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:15.969+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:16.222 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:15 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:15.969+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:16.222 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:15 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:15.969+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:16.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:15 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:15.940+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:16.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:15 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:15.940+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:16.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:15 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:15.940+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:16.520 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:16 smithi046 podman[100254]: 2024-04-17 08:04:16.158770528 +0000 UTC m=+0.826194258 container remove a9095f342142fe45ac0671672402dbc7cf8c56b0e065042530af0d05204b8d2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-17T08:04:16.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:16 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:16.354+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:16.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:16 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:16.354+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:16.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:16 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:16.354+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:16.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:16 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:16.354+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:16.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:16 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:16.505+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:16.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:16 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:16.505+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:16.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:16 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:16.505+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:16.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:16.520+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:16.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:16.520+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:16.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:16.520+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:16.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:16.520+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:17.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:16 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:16.934+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:17.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:16 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:16.934+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:17.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:16 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:16.934+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:17.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:16 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:16.963+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:17.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:16 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:16.963+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:17.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:16 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:16.963+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:17.304 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:16 smithi046 podman[100346]: 2024-04-17 08:04:16.905791566 +0000 UTC m=+0.039089723 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:17.304 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:17 smithi046 podman[100346]: 2024-04-17 08:04:17.034446545 +0000 UTC m=+0.167744736 container create cb8a4425121edd9fb81ab747de05f5051a55160603919b74ee90a7a8b6e1da55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-17T08:04:17.304 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:17 smithi046 podman[100346]: 2024-04-17 08:04:17.304978827 +0000 UTC m=+0.438277025 container init cb8a4425121edd9fb81ab747de05f5051a55160603919b74ee90a7a8b6e1da55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:04:17.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:17 smithi046 podman[100346]: 2024-04-17 08:04:17.316348044 +0000 UTC m=+0.449646237 container start cb8a4425121edd9fb81ab747de05f5051a55160603919b74ee90a7a8b6e1da55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:04:17.609 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:17 smithi046 podman[100346]: 2024-04-17 08:04:17.428777938 +0000 UTC m=+0.562076159 container attach cb8a4425121edd9fb81ab747de05f5051a55160603919b74ee90a7a8b6e1da55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-17T08:04:17.609 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:17 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:17.538+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:17.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:17 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:17.538+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:17.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:17 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:17.538+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:17.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:17 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:17.538+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:17.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:17 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:17.377+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:17.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:17 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:17.377+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:17.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:17 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:17.377+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:17.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:17 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:17.377+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:17.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:17 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:17.498+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:17.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:17 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:17.498+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:17.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:17 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:17.498+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:18.101 INFO:journalctl@ceph.osd.6.smithi046.stdout:Apr 17 08:04:17 smithi046 podman[100346]: 2024-04-17 08:04:17.788486807 +0000 UTC m=+0.921785013 container died cb8a4425121edd9fb81ab747de05f5051a55160603919b74ee90a7a8b6e1da55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-17T08:04:18.220 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:17 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:17.910+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:18.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:17 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:17.910+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:18.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:17 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:17.910+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:18.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:18 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:18.003+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:18.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:18 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:18.003+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:18.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:18 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:18.003+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:18.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.6.service' 2024-04-17T08:04:18.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:18 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:18.405+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:18.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:18 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:18.405+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:18.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:18 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:18.405+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:18.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:18 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:18.405+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:18.772 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:18.513+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:18.772 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:18.513+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:18.772 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:18.513+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:18.772 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:18 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:18.513+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:18.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:18.776 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-17T08:04:18.777 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-17T08:04:18.777 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.7 2024-04-17T08:04:18.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:18 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:18.476+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:18.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:18 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:18.476+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:18.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:18 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:18.476+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:19.104 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:18 smithi053 systemd[1]: Stopping Ceph osd.7 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:04:19.105 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:18 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:18.914+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:19.105 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:18 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:18.914+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:19.105 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:18 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:18.914+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:19.105 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:19.105+0000 7f7eb69fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:04:19.105 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:19.105+0000 7f7eb69fe700 -1 osd.7 97 *** Got signal Terminated *** 2024-04-17T08:04:19.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:19 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:19.044+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:19.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:19 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:19.044+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:19.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:19 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:19.044+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:19.423 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:19.105+0000 7f7eb69fe700 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:04:19.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:19.425+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:19.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:19.425+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:19.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:19.425+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:19.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:19.425+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:19.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:19 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:19.525+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:19.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:19 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:19.525+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:19.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:19 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:19.525+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:19.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:19 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:19.480+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:19.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:19 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:19.480+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:19.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:19 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:19.480+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:19.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:19 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:19.480+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:20.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:19.895+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:20.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:19.895+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:20.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:19 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:19.895+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:20.516 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:20 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:20.093+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:20.516 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:20 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:20.093+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:20.516 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:20 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:20.093+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:20.517 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:20 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:20.093+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:20.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:20.383+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:20.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:20.383+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:20.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:20.383+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:20.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:20.383+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:20.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:20.383+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:20.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:20 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:20.517+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:20.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:20 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:20.517+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:20.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:20 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:20.517+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:20.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:20 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:20.482+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:20.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:20 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:20.482+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:20.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:20 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:20.482+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:20.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:20 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:20.482+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:21.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:20.921+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:21.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:20.921+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:21.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:20 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:20.921+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:21.541 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:21 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:21.115+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:21.541 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:21 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:21.115+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:21.542 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:21 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:21.115+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:21.542 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:21 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:21.115+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:21.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:21.375+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:21.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:21.375+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:21.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:21.375+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:21.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:21.375+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:21.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:21.375+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:21.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:21 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:21.542+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:21.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:21 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:21.542+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:21.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:21 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:21.542+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:21.862 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:21 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:21.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:21.862 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:21 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:21.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:21.862 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:21 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:21.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:21.862 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:21 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:21.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:21.863 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:21 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:21.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:22.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:21.901+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:22.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:21.901+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:22.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:21.901+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:22.222 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:21 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:21.901+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:57.538240+0000 front 2024-04-17T08:03:57.538179+0000 (oldest deadline 2024-04-17T08:04:20.939013+0000) 2024-04-17T08:04:22.507 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:22 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:22.123+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:22.507 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:22 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:22.123+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:22.507 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:22 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:22.123+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:22.507 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:22 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:22.123+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:22.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:22.360+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:22.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:22.360+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:22.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:22.360+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:22.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:22.360+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:22.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:22.360+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:22.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:22 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:22.508+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:22.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:22 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:22.508+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:22.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:22 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:22.508+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:22.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:22 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:22.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:22.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:22 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:22.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:22.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:22 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:22.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:22.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:22 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:22.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:22.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:22 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:22.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:23.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:22.925+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:23.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:22.925+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:23.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:22.925+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:23.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:22.925+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:57.538240+0000 front 2024-04-17T08:03:57.538179+0000 (oldest deadline 2024-04-17T08:04:20.939013+0000) 2024-04-17T08:04:23.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:23.172+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:23.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:23.172+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:23.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:23.172+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:23.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:23.172+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:23.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:23.358+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:23.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:23.358+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:23.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:23.358+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:23.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:23.358+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:23.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:23.358+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:23.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:23.485+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:23.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:23.485+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:23.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:23.485+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:23.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:23 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:23.485+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:23.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:23 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:23.404+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:23.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:23 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:23.404+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:23.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:23 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:23.404+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:23.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:23 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:23.404+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:23.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:23 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:23.404+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:24.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:23.893+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:31.032405+0000 front 2024-04-17T08:03:31.032433+0000 (oldest deadline 2024-04-17T08:03:54.532530+0000) 2024-04-17T08:04:24.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:23.893+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:38.534283+0000 front 2024-04-17T08:03:38.534435+0000 (oldest deadline 2024-04-17T08:04:03.834301+0000) 2024-04-17T08:04:24.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:23.893+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.236362+0000 front 2024-04-17T08:03:47.236185+0000 (oldest deadline 2024-04-17T08:04:10.736124+0000) 2024-04-17T08:04:24.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:23 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7[54659]: 2024-04-17T08:04:23.893+0000 7f7eb2ff2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:57.538240+0000 front 2024-04-17T08:03:57.538179+0000 (oldest deadline 2024-04-17T08:04:20.939013+0000) 2024-04-17T08:04:24.450 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:24.198+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:24.450 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:24.198+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:24.450 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:24.198+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:24.450 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:24.198+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:24.570 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:24 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:24.332+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:24.571 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:24 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:24.332+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:24.571 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:24 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:24.332+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:24.571 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:24 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:24.332+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:24.571 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:24 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:24.332+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:24.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:24.451+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:24.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:24.451+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:24.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:24.451+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:24.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:24 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:24.451+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:24.854 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:24 smithi053 podman[71197]: 2024-04-17 08:04:24.572231716 +0000 UTC m=+5.577190438 container died a57362dae85109131793efaad22e15642641e599862da14a23ccb86eb1093dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-17T08:04:24.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:24.375+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:24.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:24.375+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:24.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:24.375+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:24.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:24.375+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:24.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:24 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:24.375+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:25.213 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:24 smithi053 podman[71197]: 2024-04-17 08:04:24.938776382 +0000 UTC m=+5.943735076 container cleanup a57362dae85109131793efaad22e15642641e599862da14a23ccb86eb1093dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-17T08:04:25.213 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:24 smithi053 bash[71197]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7 2024-04-17T08:04:25.470 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:25 smithi053 podman[71214]: 2024-04-17 08:04:25.215211055 +0000 UTC m=+0.638003146 container remove a57362dae85109131793efaad22e15642641e599862da14a23ccb86eb1093dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T08:04:25.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:25 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:25.297+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:25.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:25 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:25.297+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:25.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:25 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:25.297+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:25.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:25 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:25.297+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:25.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:25 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:25.297+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:25.552 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:25.438+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:25.552 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:25.438+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:25.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:25.438+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:25.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:25.438+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:25.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:25.236+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:25.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:25.236+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:25.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:25.236+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:25.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:25 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:25.236+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:25.834 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:25 smithi053 podman[71304]: 2024-04-17 08:04:25.420645406 +0000 UTC m=+0.018518689 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:25.834 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:25 smithi053 podman[71304]: 2024-04-17 08:04:25.532817858 +0000 UTC m=+0.130691118 container create f9ac40cd3c971055a31f0c67e3ea830137ee8a0890c2513374d130da598bc02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-17T08:04:25.834 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:25 smithi053 podman[71304]: 2024-04-17 08:04:25.743783747 +0000 UTC m=+0.341657022 container init f9ac40cd3c971055a31f0c67e3ea830137ee8a0890c2513374d130da598bc02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:25.834 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:25 smithi053 podman[71304]: 2024-04-17 08:04:25.74691152 +0000 UTC m=+0.344784785 container start f9ac40cd3c971055a31f0c67e3ea830137ee8a0890c2513374d130da598bc02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5) 2024-04-17T08:04:25.835 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:25 smithi053 podman[71304]: 2024-04-17 08:04:25.835622615 +0000 UTC m=+0.433495882 container attach f9ac40cd3c971055a31f0c67e3ea830137ee8a0890c2513374d130da598bc02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-17T08:04:25.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:25.408+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:25.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:25.408+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:25.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:25.408+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:25.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:25.408+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:25.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:25 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:25.408+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:26.221 INFO:journalctl@ceph.osd.7.smithi053.stdout:Apr 17 08:04:26 smithi053 podman[71304]: 2024-04-17 08:04:26.04827746 +0000 UTC m=+0.646150726 container died f9ac40cd3c971055a31f0c67e3ea830137ee8a0890c2513374d130da598bc02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-17T08:04:26.552 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:26.481+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:26.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:26.481+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:26.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:26.481+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:26.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:26.481+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:26.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:26.255+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:26.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:26.255+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:26.554 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:26.255+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:26.554 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:26.255+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:26.554 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:26 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:26.259+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:26.554 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:26 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:26.259+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:26.555 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:26 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:26.259+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:26.555 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:26 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:26.259+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:26.555 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:26 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:26.259+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:26.572 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.7.service' 2024-04-17T08:04:26.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:26.618 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-17T08:04:26.618 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-17T08:04:26.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.8 2024-04-17T08:04:26.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:26 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:26.387+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:26.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:26 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:26.387+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:26.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:26 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:26.387+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:26.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:26 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:26.387+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:26.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:26 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:26.387+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:26.922 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 systemd[1]: Stopping Ceph osd.8 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:04:26.923 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:26.923+0000 7f2c2fe42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:04:26.923 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:26.923+0000 7f2c2fe42700 -1 osd.8 97 *** Got signal Terminated *** 2024-04-17T08:04:27.221 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:26 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:26.923+0000 7f2c2fe42700 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:04:27.479 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:27.222+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:27.479 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:27.222+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:27.480 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:27.222+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:27.480 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:27.222+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:27.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:27.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:27.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:27.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:27.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:27.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:27.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:27.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:27.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:27.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:27.762 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:27.370+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:27.762 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:27.370+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:27.762 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:27.370+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:27.762 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:27.370+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:27.762 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:27 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:27.370+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:27.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:27.480+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:27.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:27.480+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:27.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:27.480+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:27.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:27 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:27.480+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:28.552 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:28.474+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:28.552 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:28.474+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:28.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:28.474+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:28.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:28.474+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:28.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:28.263+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:28.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:28.263+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:28.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:28.263+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:28.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:28.263+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:28.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:28 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:28.263+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:28.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:28 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:28.266+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:28.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:28 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:28.267+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:28.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:28 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:28.267+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:28.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:28 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:28.267+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:28.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:28 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:28.267+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:28.798 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:28.361+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:28.798 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:28.361+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:28.799 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:28.361+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:28.799 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:28.361+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:28.799 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:28 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:28.361+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:29.494 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:29 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:29.366+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:29.495 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:29 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:29.366+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:29.495 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:29 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:29.366+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:29.495 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:29 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:29.366+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:29.495 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:29 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:29.366+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:29.495 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:29 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:29.366+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:29.552 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:29.499+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:29.552 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:29.499+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:29.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:29.499+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:29.553 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:29.499+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:29.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:29.281+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:29.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:29.281+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:29.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:29.281+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:29.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:29.281+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:29.554 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:29 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:29.281+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:29.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:29 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:29.231+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:29.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:29 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:29.231+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:29.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:29 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:29.231+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:29.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:29 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:29.231+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:29.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:29 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:29.231+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:29.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:29 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:29.231+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:30.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:30 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:30.242+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:30.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:30 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:30.242+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:30.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:30 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:30.242+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:30.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:30 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:30.242+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:30.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:30 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:30.242+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:30.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:30 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:30.242+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:30.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:30.513+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:30.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:30.513+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:30.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:30.513+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:30.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:30.513+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:30.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:30.513+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.928445+0000 front 2024-04-17T08:04:05.928472+0000 (oldest deadline 2024-04-17T08:04:30.028461+0000) 2024-04-17T08:04:30.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:30.312+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:30.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:30.312+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:30.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:30.312+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:30.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:30.312+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:30.804 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:30 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:30.312+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:30.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:30.367+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:30.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:30.367+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:30.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:30.367+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:30.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:30.367+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:30.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:30.367+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:30.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:30 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:30.367+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:31.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:31.355+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:31.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:31.355+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:31.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:31.355+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:31.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:31.355+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:31.611 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:31.355+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:31.611 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:31 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:31.355+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:31.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:31.279+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:31.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:31.279+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:31.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:31.279+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:31.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:31.279+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:31.722 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:31.279+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:31.722 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:31 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:31.279+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:31.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:31.515+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.522104+0000 front 2024-04-17T08:03:30.522215+0000 (oldest deadline 2024-04-17T08:03:55.122557+0000) 2024-04-17T08:04:31.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:31.515+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.823531+0000 front 2024-04-17T08:03:39.823605+0000 (oldest deadline 2024-04-17T08:04:02.123665+0000) 2024-04-17T08:04:31.802 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:31.515+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:48.525661+0000 front 2024-04-17T08:03:48.525511+0000 (oldest deadline 2024-04-17T08:04:10.225323+0000) 2024-04-17T08:04:31.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:31.515+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:58.427111+0000 front 2024-04-17T08:03:58.426983+0000 (oldest deadline 2024-04-17T08:04:22.527092+0000) 2024-04-17T08:04:31.803 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8[55398]: 2024-04-17T08:04:31.515+0000 7f2c2842e700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.928445+0000 front 2024-04-17T08:04:05.928472+0000 (oldest deadline 2024-04-17T08:04:30.028461+0000) 2024-04-17T08:04:31.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:31.355+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:31.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:31.355+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:31.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:31.355+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:31.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:31.355+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:31.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:31 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:31.355+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:32.641 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:32 smithi062 podman[72003]: 2024-04-17 08:04:32.327001188 +0000 UTC m=+5.506968002 container died 4592ca009bfa1f06b28c6aae7e55b9e753239860237500e859425621ca1f11e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:32.641 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:32 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:32.377+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:32.641 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:32 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:32.377+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:32.641 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:32 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:32.377+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:32.641 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:32 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:32.377+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:32.641 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:32 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:32.377+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:32.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:32 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:32.290+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:32.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:32 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:32.290+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:32.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:32 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:32.290+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:32.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:32 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:32.290+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:32.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:32 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:32.290+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:32.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:32 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:32.290+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:32.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:32 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:32.395+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:32.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:32 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:32.395+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:32.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:32 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:32.395+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:32.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:32 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:32.395+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:32.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:32 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:32.395+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:32.861 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:32 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:32.395+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:32.991 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:32 smithi062 podman[72003]: 2024-04-17 08:04:32.740987841 +0000 UTC m=+5.920954641 container cleanup 4592ca009bfa1f06b28c6aae7e55b9e753239860237500e859425621ca1f11e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:04:32.991 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:32 smithi062 bash[72003]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8 2024-04-17T08:04:33.302 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:32 smithi062 podman[72020]: 2024-04-17 08:04:32.992397822 +0000 UTC m=+0.667961536 container remove 4592ca009bfa1f06b28c6aae7e55b9e753239860237500e859425621ca1f11e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-17T08:04:33.702 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:33 smithi062 podman[72114]: 2024-04-17 08:04:33.31203523 +0000 UTC m=+0.018202584 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:33.703 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:33 smithi062 podman[72114]: 2024-04-17 08:04:33.42684826 +0000 UTC m=+0.133015615 container create c5b2ba6fa88d8694b943d2c8fd43b87acc075dc3d88e91832dc003e3acc97ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416) 2024-04-17T08:04:33.703 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:33 smithi062 podman[72114]: 2024-04-17 08:04:33.704337238 +0000 UTC m=+0.410504574 container init c5b2ba6fa88d8694b943d2c8fd43b87acc075dc3d88e91832dc003e3acc97ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T08:04:33.703 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:33 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:33.336+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:33.703 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:33 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:33.336+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:33.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:33 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:33.336+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:33.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:33 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:33.336+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:33.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:33 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:33.336+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:33.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:33.281+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:33.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:33.281+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:33.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:33.281+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:33.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:33.281+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:33.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:33.281+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:33.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:33.281+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:33.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:33.410+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:33.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:33.410+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:33.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:33.410+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:33.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:33.410+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:33.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:33.410+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:33.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:33 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:33.410+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:34.006 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:33 smithi062 podman[72114]: 2024-04-17 08:04:33.70787288 +0000 UTC m=+0.414040208 container start c5b2ba6fa88d8694b943d2c8fd43b87acc075dc3d88e91832dc003e3acc97ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-17T08:04:34.006 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:33 smithi062 podman[72114]: 2024-04-17 08:04:33.796081601 +0000 UTC m=+0.502248948 container attach c5b2ba6fa88d8694b943d2c8fd43b87acc075dc3d88e91832dc003e3acc97ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:34.272 INFO:journalctl@ceph.osd.8.smithi062.stdout:Apr 17 08:04:34 smithi062 podman[72114]: 2024-04-17 08:04:34.007961278 +0000 UTC m=+0.714128628 container died c5b2ba6fa88d8694b943d2c8fd43b87acc075dc3d88e91832dc003e3acc97ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True) 2024-04-17T08:04:34.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:34 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:34.303+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:34.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:34 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:34.303+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:34.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:34 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:34.303+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:34.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:34 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:34.303+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:34.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:34 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:34.303+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:34.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.8.service' 2024-04-17T08:04:34.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:34.666 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-17T08:04:34.667 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-17T08:04:34.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.9 2024-04-17T08:04:34.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:34 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:34.330+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:34.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:34 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:34.330+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:34.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:34 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:34.330+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:34.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:34 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:34.330+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:34.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:34 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:34.330+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:34.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:34 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:34.330+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:34.760 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:34 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:34.441+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:34.760 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:34 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:34.441+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:34.760 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:34 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:34.441+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:34.760 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:34 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:34.441+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:34.760 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:34 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:34.441+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:34.761 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:34 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:34.441+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:35.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:34 smithi046 systemd[1]: Stopping Ceph osd.9 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:04:35.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:35.353+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:35.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:35.353+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:35.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:35.353+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:35.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:35.353+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:35.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:35 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:35.353+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:35.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.299+0000 7f951d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:04:35.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.299+0000 7f951d800700 -1 osd.9 97 *** Got signal Terminated *** 2024-04-17T08:04:35.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.299+0000 7f951d800700 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:04:35.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:35.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:35.610 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:35.611 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:35.611 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:35.611 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:35 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:35.432+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:35.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:35 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:35.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:35.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:35 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:35.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:35.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:35 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:35.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:35.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:35 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:35.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:35.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:35 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:35.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:35.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:35 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:35.348+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:36.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:36.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:36.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:36.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:36.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:36.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:36.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:36.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:36.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:36.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:36.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:36.347+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:36.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:36 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:36.331+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:36.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:36 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:36.331+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:36.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:36 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:36.331+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:36.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:36 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:36.331+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:36.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:36 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:36.331+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:36.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:36 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:36.332+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:36.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:36.397+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:36.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:36.397+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:36.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:36.397+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:36.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:36.397+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:36.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:36.397+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:36.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:36 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:36.397+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:37.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:37 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:37.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:37.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:37 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:37.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:37.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:37 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:37.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:37.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:37 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:37.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:37.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:37 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:37.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:37.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:37 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:37.305+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:37.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:37 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:37.374+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:37.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:37 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:37.374+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:37.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:37 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:37.374+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:37.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:37 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:37.374+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:37.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:37 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:37.374+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:37.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:37 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:37.374+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:37.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:37.440+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:37.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:37.440+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:37.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:37.440+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:37.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:37.440+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:37.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:37.440+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:37.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:37 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:37.440+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:38.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:38.291+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:38.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:38.291+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:38.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:38.291+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:38.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:38.291+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:38.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:38.291+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:38.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:38.291+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:38.722 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:38 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:38.291+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:38.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:38.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:38.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:38.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:38.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:38.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:38.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:38.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:38.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:38.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:38.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:38.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:38.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:38.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:38.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:38.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:38.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:38.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:38.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:38.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:38.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:38.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:38.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:38 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:38.449+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:39.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:39.286+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:39.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:39.286+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:39.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:39.286+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:39.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:39.286+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:39.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:39.286+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:39.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:39.286+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:39.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:39 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:39.286+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:39.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:39 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:39.380+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:39.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:39 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:39.380+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:39.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:39 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:39.380+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:39.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:39 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:39.380+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:39.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:39 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:39.380+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:39.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:39 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:39.380+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:39.859 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:39.406+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:22.139497+0000 front 2024-04-17T08:03:22.139978+0000 (oldest deadline 2024-04-17T08:03:45.639742+0000) 2024-04-17T08:04:39.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:39.406+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:32.544316+0000 front 2024-04-17T08:03:32.544550+0000 (oldest deadline 2024-04-17T08:03:57.844486+0000) 2024-04-17T08:04:39.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:39.406+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:40.746599+0000 front 2024-04-17T08:03:40.746410+0000 (oldest deadline 2024-04-17T08:04:03.046644+0000) 2024-04-17T08:04:39.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:39.406+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:47.748316+0000 front 2024-04-17T08:03:47.749044+0000 (oldest deadline 2024-04-17T08:04:12.448359+0000) 2024-04-17T08:04:39.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:39.406+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.350414+0000 front 2024-04-17T08:03:55.350474+0000 (oldest deadline 2024-04-17T08:04:21.250655+0000) 2024-04-17T08:04:39.860 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:39 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9[66916]: 2024-04-17T08:04:39.406+0000 7f9519000700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.154590+0000 front 2024-04-17T08:04:05.154376+0000 (oldest deadline 2024-04-17T08:04:28.654369+0000) 2024-04-17T08:04:40.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:40 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:40.250+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:40.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:40 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:40.250+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:40.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:40 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:40.250+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:40.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:40 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:40.250+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:40.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:40 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:40.250+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:40.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:40 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:40.250+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:40.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:40 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:40.250+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:40.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:40 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:40.346+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:40.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:40 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:40.346+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:40.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:40 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:40.346+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:40.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:40 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:40.346+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:40.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:40 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:40.346+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:40.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:40 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:40.346+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:41.185 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:40 smithi046 podman[100462]: 2024-04-17 08:04:40.875590571 +0000 UTC m=+5.756755606 container died aa0206653c5d8efb78b85c7dcc012ab23c9a2a3293e03ad242aea42a2e86317e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-17T08:04:41.509 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:41 smithi046 podman[100462]: 2024-04-17 08:04:41.292872444 +0000 UTC m=+6.174037486 container cleanup aa0206653c5d8efb78b85c7dcc012ab23c9a2a3293e03ad242aea42a2e86317e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-17T08:04:41.510 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:41 smithi046 bash[100462]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9 2024-04-17T08:04:41.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:41.369+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:41.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:41.369+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:41.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:41.369+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:41.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:41.369+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:41.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:41.369+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:41.704 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:41 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:41.369+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:41.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:41 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:41.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:41.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:41 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:41.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:41.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:41 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:41.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:41.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:41 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:41.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:41.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:41 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:41.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:41.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:41 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:41.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:41.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:41 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:41.255+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:41.858 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:41 smithi046 podman[100482]: 2024-04-17 08:04:41.510413564 +0000 UTC m=+0.620170838 container remove aa0206653c5d8efb78b85c7dcc012ab23c9a2a3293e03ad242aea42a2e86317e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:04:42.690 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:42 smithi046 podman[100568]: 2024-04-17 08:04:42.330157453 +0000 UTC m=+0.039207613 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:42.690 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:42 smithi046 podman[100568]: 2024-04-17 08:04:42.436264288 +0000 UTC m=+0.145314485 container create 29f3a59edb72d5d8c6ae6f3ca4624cdae48fdddcf3067b98ac94e1201031fff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-17T08:04:42.691 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:42 smithi046 podman[100568]: 2024-04-17 08:04:42.689641568 +0000 UTC m=+0.398691788 container init 29f3a59edb72d5d8c6ae6f3ca4624cdae48fdddcf3067b98ac94e1201031fff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-17T08:04:42.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:42 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:42.260+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:42.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:42 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:42.260+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:42.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:42 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:42.260+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:42.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:42 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:42.260+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:42.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:42 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:42.260+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:42.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:42 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:42.260+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:42.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:42 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:42.260+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:42.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:42.351+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:42.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:42.351+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:42.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:42.351+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:42.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:42.351+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:42.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:42.351+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:42.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:42 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:42.351+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:43.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:42 smithi046 podman[100568]: 2024-04-17 08:04:42.701352878 +0000 UTC m=+0.410403078 container start 29f3a59edb72d5d8c6ae6f3ca4624cdae48fdddcf3067b98ac94e1201031fff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:43.109 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:42 smithi046 podman[100568]: 2024-04-17 08:04:42.788828186 +0000 UTC m=+0.497878389 container attach 29f3a59edb72d5d8c6ae6f3ca4624cdae48fdddcf3067b98ac94e1201031fff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-17T08:04:43.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:43 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:43.216+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:43.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:43 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:43.216+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:43.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:43 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:43.216+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:43.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:43 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:43.216+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:43.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:43 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:43.216+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:43.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:43 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:43.216+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:43.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:43 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:43.216+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:43.544 INFO:journalctl@ceph.osd.9.smithi046.stdout:Apr 17 08:04:43 smithi046 podman[100594]: 2024-04-17 08:04:43.227034486 +0000 UTC m=+0.043436568 container died 29f3a59edb72d5d8c6ae6f3ca4624cdae48fdddcf3067b98ac94e1201031fff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-17T08:04:43.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:43.352+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:43.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:43.352+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:43.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:43.352+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:43.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:43.352+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:43.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:43.352+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:43.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:43.352+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:43.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.9.service' 2024-04-17T08:04:44.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:44.110 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-17T08:04:44.110 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-17T08:04:44.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.10 2024-04-17T08:04:44.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 systemd[1]: Stopping Ceph osd.10 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:04:44.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.224+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:44.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.224+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:44.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.224+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:44.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.224+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:44.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.224+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:44.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.224+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:44.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.224+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:44.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.391+0000 7f294ce11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:04:44.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.391+0000 7f294ce11700 -1 osd.10 97 *** Got signal Terminated *** 2024-04-17T08:04:44.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:44 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:44.391+0000 7f294ce11700 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:04:44.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:44.307+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:44.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:44.307+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:44.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:44.307+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:44.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:44.307+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:44.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:44.307+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:44.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:44 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:44.307+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:45.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:45.218+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:45.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:45.218+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:45.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:45.218+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:45.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:45.218+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:45.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:45.218+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:45.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:45.218+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:45.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:45 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:45.218+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:45.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:45.358+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:45.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:45.358+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:45.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:45.358+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:45.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:45.358+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:45.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:45.358+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:45.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:45 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:45.358+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:46.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:46.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:46.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:46.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:46.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:46.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:46.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:46.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:46.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:46.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:46.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:46.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:46.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:46 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:46.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:46.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:46.327+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:47.060 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:46.327+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:47.060 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:46.327+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:47.061 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:46.327+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:47.061 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:46.327+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:47.061 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:46 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:46.327+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:47.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:47.179+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:47.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:47.179+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:47.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:47.179+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:47.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:47.179+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:47.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:47.179+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:47.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:47.179+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:47.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:47 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:47.179+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:47.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:47.297+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:47.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:47.297+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:47.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:47.297+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:47.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:47.297+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:47.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:47.297+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:47.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:47 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:47.297+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:48.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:48.253+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:48.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:48.253+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:48.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:48.253+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:48.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:48.253+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:48.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:48.253+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:48.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:48.253+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:48.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:48.253+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:48.720 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:48.223+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:48.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:48.223+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:48.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:48.223+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:48.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:48.223+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:48.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:48.223+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:48.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:48.223+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:48.721 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:48 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:48.223+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:49.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:49.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.968321+0000 front 2024-04-17T08:03:20.968393+0000 (oldest deadline 2024-04-17T08:03:45.067730+0000) 2024-04-17T08:04:49.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:49.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.968463+0000 front 2024-04-17T08:03:30.968575+0000 (oldest deadline 2024-04-17T08:03:55.068677+0000) 2024-04-17T08:04:49.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:49.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:39.069940+0000 front 2024-04-17T08:03:39.069819+0000 (oldest deadline 2024-04-17T08:04:03.769950+0000) 2024-04-17T08:04:49.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:49.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-04-17T08:03:46.070990+0000 front 2024-04-17T08:03:46.071140+0000 (oldest deadline 2024-04-17T08:04:11.970869+0000) 2024-04-17T08:04:49.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:49.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:55.471538+0000 front 2024-04-17T08:03:55.471507+0000 (oldest deadline 2024-04-17T08:04:19.571630+0000) 2024-04-17T08:04:49.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:49.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:04.173100+0000 front 2024-04-17T08:04:04.173078+0000 (oldest deadline 2024-04-17T08:04:28.873044+0000) 2024-04-17T08:04:49.472 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10[61700]: 2024-04-17T08:04:49.208+0000 7f2949405700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:13.375009+0000 front 2024-04-17T08:04:13.375072+0000 (oldest deadline 2024-04-17T08:04:37.474772+0000) 2024-04-17T08:04:49.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:49.249+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:49.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:49.249+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:49.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:49.249+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:49.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:49.249+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:49.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:49.249+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:49.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:49.249+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:49.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:49 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:49.249+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:50.157 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:49 smithi053 podman[71408]: 2024-04-17 08:04:49.881893729 +0000 UTC m=+5.575637720 container died d0d883f9eb47bfa923564f9bc1647da1a60d00ff75094dd5decc779167e206f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True) 2024-04-17T08:04:50.470 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:50 smithi053 podman[71408]: 2024-04-17 08:04:50.249841382 +0000 UTC m=+5.943585370 container cleanup d0d883f9eb47bfa923564f9bc1647da1a60d00ff75094dd5decc779167e206f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:50.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:50 smithi053 bash[71408]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10 2024-04-17T08:04:50.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:50.291+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:50.552 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:50.291+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:50.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:50.291+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:50.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:50.291+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:50.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:50.291+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:50.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:50.291+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:50.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:50 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:50.291+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:50.808 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:50 smithi053 podman[71426]: 2024-04-17 08:04:50.501006104 +0000 UTC m=+0.608577259 container remove d0d883f9eb47bfa923564f9bc1647da1a60d00ff75094dd5decc779167e206f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T08:04:51.102 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:50 smithi053 podman[71514]: 2024-04-17 08:04:50.810012828 +0000 UTC m=+0.102069059 container create 44ef316af3598e355d41bbb63d8997b4bd2d22a95f8d752df3473ce2d11a6f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:04:51.102 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:50 smithi053 podman[71514]: 2024-04-17 08:04:50.726318924 +0000 UTC m=+0.018375178 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:51.103 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 podman[71514]: 2024-04-17 08:04:51.029211324 +0000 UTC m=+0.321267561 container init 44ef316af3598e355d41bbb63d8997b4bd2d22a95f8d752df3473ce2d11a6f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-17T08:04:51.103 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 podman[71514]: 2024-04-17 08:04:51.0324843 +0000 UTC m=+0.324540536 container start 44ef316af3598e355d41bbb63d8997b4bd2d22a95f8d752df3473ce2d11a6f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-17T08:04:51.103 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 podman[71514]: 2024-04-17 08:04:51.104036379 +0000 UTC m=+0.396092631 container attach 44ef316af3598e355d41bbb63d8997b4bd2d22a95f8d752df3473ce2d11a6f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-17T08:04:51.471 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 podman[71514]: 2024-04-17 08:04:51.333323735 +0000 UTC m=+0.625379988 container died 44ef316af3598e355d41bbb63d8997b4bd2d22a95f8d752df3473ce2d11a6f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:04:51.762 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:51.306+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:51.762 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:51.306+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:51.762 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:51.306+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:51.763 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:51.306+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:51.763 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:51.306+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:51.763 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:51.306+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:51.763 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:51.306+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:51.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.10.service' 2024-04-17T08:04:51.874 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 podman[71540]: 2024-04-17 08:04:51.824162689 +0000 UTC m=+0.485544028 container remove 44ef316af3598e355d41bbb63d8997b4bd2d22a95f8d752df3473ce2d11a6f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-17T08:04:51.874 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 systemd[1]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-17T08:04:51.874 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 systemd[1]: Stopped Ceph osd.10 for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T08:04:51.874 INFO:journalctl@ceph.osd.10.smithi053.stdout:Apr 17 08:04:51 smithi053 systemd[1]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.10.service: Consumed 10.857s CPU time. 2024-04-17T08:04:51.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:51.890 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-17T08:04:51.891 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-17T08:04:51.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.11 2024-04-17T08:04:52.052 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:51 smithi062 systemd[1]: Stopping Ceph osd.11 for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:04:52.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.193+0000 7f9f00eb8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:04:52.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.193+0000 7f9f00eb8700 -1 osd.11 97 *** Got signal Terminated *** 2024-04-17T08:04:52.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.193+0000 7f9f00eb8700 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T08:04:52.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:52.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:52.553 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:52.554 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:52.554 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:52.554 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:52.554 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:52 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:52.344+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:53.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:53.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:53.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:53.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:53.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:53.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:53.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:53.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:53 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:53.381+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6820 osd.8 since back 2024-04-17T08:04:27.932730+0000 front 2024-04-17T08:04:27.932773+0000 (oldest deadline 2024-04-17T08:04:52.632838+0000) 2024-04-17T08:04:54.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:54.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:54.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:54.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:54.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:54.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:54.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:54.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:54 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:54.342+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6820 osd.8 since back 2024-04-17T08:04:27.932730+0000 front 2024-04-17T08:04:27.932773+0000 (oldest deadline 2024-04-17T08:04:52.632838+0000) 2024-04-17T08:04:55.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:55.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:55.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:55.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:55.804 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:55.804 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:55.804 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:55.804 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:55 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:55.359+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6820 osd.8 since back 2024-04-17T08:04:27.932730+0000 front 2024-04-17T08:04:27.932773+0000 (oldest deadline 2024-04-17T08:04:52.632838+0000) 2024-04-17T08:04:56.802 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-04-17T08:03:20.922010+0000 front 2024-04-17T08:03:20.921936+0000 (oldest deadline 2024-04-17T08:03:45.621846+0000) 2024-04-17T08:04:56.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-04-17T08:03:30.823602+0000 front 2024-04-17T08:03:30.823575+0000 (oldest deadline 2024-04-17T08:03:54.323637+0000) 2024-04-17T08:04:56.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6804 osd.2 since back 2024-04-17T08:03:36.624613+0000 front 2024-04-17T08:03:36.624588+0000 (oldest deadline 2024-04-17T08:04:01.324818+0000) 2024-04-17T08:04:56.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-04-17T08:03:52.927717+0000 front 2024-04-17T08:03:52.927835+0000 (oldest deadline 2024-04-17T08:04:19.928027+0000) 2024-04-17T08:04:56.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6812 osd.5 since back 2024-04-17T08:04:05.228981+0000 front 2024-04-17T08:04:05.228934+0000 (oldest deadline 2024-04-17T08:04:27.529125+0000) 2024-04-17T08:04:56.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-04-17T08:04:11.030273+0000 front 2024-04-17T08:04:11.030205+0000 (oldest deadline 2024-04-17T08:04:35.730114+0000) 2024-04-17T08:04:56.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-04-17T08:04:23.232381+0000 front 2024-04-17T08:04:23.232319+0000 (oldest deadline 2024-04-17T08:04:47.932193+0000) 2024-04-17T08:04:56.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:56 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11[62492]: 2024-04-17T08:04:56.390+0000 7f9ef94a4700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.62:6820 osd.8 since back 2024-04-17T08:04:27.932730+0000 front 2024-04-17T08:04:27.932773+0000 (oldest deadline 2024-04-17T08:04:52.632838+0000) 2024-04-17T08:04:57.803 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:57 smithi062 podman[72231]: 2024-04-17 08:04:57.527228942 +0000 UTC m=+5.438967207 container died 29002b1f2e1c6c6e3e1238336c31ad6df113b3a272826612eeeca5d2a25c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-17T08:04:58.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:57 smithi062 podman[72231]: 2024-04-17 08:04:57.818914325 +0000 UTC m=+5.730652556 container cleanup 29002b1f2e1c6c6e3e1238336c31ad6df113b3a272826612eeeca5d2a25c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-17T08:04:58.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:57 smithi062 bash[72231]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11 2024-04-17T08:04:58.303 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:58 smithi062 podman[72248]: 2024-04-17 08:04:58.045346985 +0000 UTC m=+0.516382683 container remove 29002b1f2e1c6c6e3e1238336c31ad6df113b3a272826612eeeca5d2a25c98df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:04:58.688 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:58 smithi062 podman[72334]: 2024-04-17 08:04:58.249112917 +0000 UTC m=+0.018461558 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T08:04:58.688 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:58 smithi062 podman[72334]: 2024-04-17 08:04:58.407658544 +0000 UTC m=+0.177007164 container create 7e30cca74195f7c00c90a77602dbe584063aa57564ef74c796e85d2e81e32a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:04:58.689 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:58 smithi062 podman[72334]: 2024-04-17 08:04:58.690354677 +0000 UTC m=+0.459703313 container init 7e30cca74195f7c00c90a77602dbe584063aa57564ef74c796e85d2e81e32a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-17T08:04:58.994 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:58 smithi062 podman[72334]: 2024-04-17 08:04:58.693613917 +0000 UTC m=+0.462962546 container start 7e30cca74195f7c00c90a77602dbe584063aa57564ef74c796e85d2e81e32a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-17T08:04:58.995 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:58 smithi062 podman[72334]: 2024-04-17 08:04:58.890232572 +0000 UTC m=+0.659581222 container attach 7e30cca74195f7c00c90a77602dbe584063aa57564ef74c796e85d2e81e32a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-17T08:04:59.302 INFO:journalctl@ceph.osd.11.smithi062.stdout:Apr 17 08:04:58 smithi062 podman[72334]: 2024-04-17 08:04:58.996417127 +0000 UTC m=+0.765765754 container died 7e30cca74195f7c00c90a77602dbe584063aa57564ef74c796e85d2e81e32a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:04:59.466 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@osd.11.service' 2024-04-17T08:04:59.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:04:59.511 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-17T08:04:59.512 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-17T08:04:59.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.a 2024-04-17T08:04:59.859 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 08:04:59 smithi046 systemd[1]: Stopping Ceph mds.a for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:00.359 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 08:05:00 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a[73864]: 2024-04-17T08:05:00.101+0000 7fb1ad800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:00.359 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 08:05:00 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a[73864]: 2024-04-17T08:05:00.101+0000 7fb1ad800700 -1 mds.a *** got signal Terminated *** 2024-04-17T08:05:04.399 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 08:05:04 smithi046 podman[100685]: 2024-04-17 08:05:04.121449859 +0000 UTC m=+4.149709774 container died bbe75030bc0aadd83f82c2df6a111d63fbd475aae88eaa5024a48081e1183c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:05:04.718 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 08:05:04 smithi046 podman[100685]: 2024-04-17 08:05:04.510345148 +0000 UTC m=+4.538605057 container cleanup bbe75030bc0aadd83f82c2df6a111d63fbd475aae88eaa5024a48081e1183c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:05:04.719 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 08:05:04 smithi046 bash[100685]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a 2024-04-17T08:05:05.109 INFO:journalctl@ceph.mds.a.smithi046.stdout:Apr 17 08:05:04 smithi046 podman[100710]: 2024-04-17 08:05:04.719525562 +0000 UTC m=+0.582021767 container remove bbe75030bc0aadd83f82c2df6a111d63fbd475aae88eaa5024a48081e1183c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-17T08:05:05.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.a.service' 2024-04-17T08:05:05.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:05.382 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-17T08:05:05.383 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-17T08:05:05.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.d 2024-04-17T08:05:05.859 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 08:05:05 smithi046 systemd[1]: Stopping Ceph mds.d for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:06.359 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 08:05:05 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d[72831]: 2024-04-17T08:05:05.977+0000 7f5827400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:06.359 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 08:05:05 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d[72831]: 2024-04-17T08:05:05.977+0000 7f5827400700 -1 mds.d *** got signal Terminated *** 2024-04-17T08:05:09.609 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 08:05:09 smithi046 podman[100815]: 2024-04-17 08:05:09.317167116 +0000 UTC m=+3.453137233 container died af92cf60c6f9e04b3cd6a579d402bba13c0dc798a403ff3f8916bd92659af4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-17T08:05:09.997 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 08:05:09 smithi046 podman[100815]: 2024-04-17 08:05:09.73026065 +0000 UTC m=+3.866230787 container cleanup af92cf60c6f9e04b3cd6a579d402bba13c0dc798a403ff3f8916bd92659af4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-17T08:05:09.998 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 08:05:09 smithi046 bash[100815]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d 2024-04-17T08:05:10.359 INFO:journalctl@ceph.mds.d.smithi046.stdout:Apr 17 08:05:09 smithi046 podman[100827]: 2024-04-17 08:05:09.99822165 +0000 UTC m=+0.665110512 container remove af92cf60c6f9e04b3cd6a579d402bba13c0dc798a403ff3f8916bd92659af4d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T08:05:10.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.d.service' 2024-04-17T08:05:10.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:10.668 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-17T08:05:10.668 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-17T08:05:10.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.g 2024-04-17T08:05:11.108 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:05:10 smithi046 systemd[1]: Stopping Ceph mds.g for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:11.609 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:05:11 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g[73480]: 2024-04-17T08:05:11.229+0000 7fd3d4600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:11.609 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:05:11 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g[73480]: 2024-04-17T08:05:11.229+0000 7fd3d4600700 -1 mds.g *** got signal Terminated *** 2024-04-17T08:05:15.187 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:05:14 smithi046 podman[100935]: 2024-04-17 08:05:14.922776722 +0000 UTC m=+3.821831592 container died b88ece623d38b12f41c9fc2efa212f345b5edface0aa953d78516eb6f4fac0f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-17T08:05:15.517 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:05:15 smithi046 podman[100935]: 2024-04-17 08:05:15.300221873 +0000 UTC m=+4.199276753 container cleanup b88ece623d38b12f41c9fc2efa212f345b5edface0aa953d78516eb6f4fac0f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-17T08:05:15.517 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:05:15 smithi046 bash[100935]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g 2024-04-17T08:05:15.859 INFO:journalctl@ceph.mds.g.smithi046.stdout:Apr 17 08:05:15 smithi046 podman[100949]: 2024-04-17 08:05:15.517805542 +0000 UTC m=+0.581048837 container remove b88ece623d38b12f41c9fc2efa212f345b5edface0aa953d78516eb6f4fac0f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-17T08:05:16.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.g.service' 2024-04-17T08:05:16.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:16.189 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-17T08:05:16.189 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-17T08:05:16.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.j 2024-04-17T08:05:16.609 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 08:05:16 smithi046 systemd[1]: Stopping Ceph mds.j for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:17.109 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 08:05:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-j[72495]: 2024-04-17T08:05:16.776+0000 7f683c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:17.110 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 08:05:16 smithi046 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-j[72495]: 2024-04-17T08:05:16.776+0000 7f683c000700 -1 mds.j *** got signal Terminated *** 2024-04-17T08:05:21.109 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 08:05:20 smithi046 podman[101055]: 2024-04-17 08:05:20.793310082 +0000 UTC m=+4.147736803 container died 65861081c639ebf9d39c2c1887a1a121245dd76dec0bee383886bd08dd783b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:05:21.504 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 08:05:21 smithi046 podman[101055]: 2024-04-17 08:05:21.320292488 +0000 UTC m=+4.674719184 container cleanup 65861081c639ebf9d39c2c1887a1a121245dd76dec0bee383886bd08dd783b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-17T08:05:21.504 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 08:05:21 smithi046 bash[101055]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-j 2024-04-17T08:05:21.859 INFO:journalctl@ceph.mds.j.smithi046.stdout:Apr 17 08:05:21 smithi046 podman[101081]: 2024-04-17 08:05:21.504568578 +0000 UTC m=+0.692462194 container remove 65861081c639ebf9d39c2c1887a1a121245dd76dec0bee383886bd08dd783b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:05:22.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.j.service' 2024-04-17T08:05:22.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:22.144 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-17T08:05:22.144 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-17T08:05:22.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.b 2024-04-17T08:05:22.448 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:05:22 smithi053 systemd[1]: Stopping Ceph mds.b for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:22.720 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:05:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: 2024-04-17T08:05:22.449+0000 7fd05be0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:22.720 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:05:22 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b[67393]: 2024-04-17T08:05:22.449+0000 7fd05be0d700 -1 mds.b *** got signal Terminated *** 2024-04-17T08:05:26.720 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:05:26 smithi053 podman[71620]: 2024-04-17 08:05:26.458094472 +0000 UTC m=+4.108432418 container died 98169a4dcdb2cc5f13a834a68b79630f75b4a35d46cf903989a2b91e31d82e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-17T08:05:27.140 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:05:26 smithi053 podman[71620]: 2024-04-17 08:05:26.940891843 +0000 UTC m=+4.591229751 container cleanup 98169a4dcdb2cc5f13a834a68b79630f75b4a35d46cf903989a2b91e31d82e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-17T08:05:27.140 INFO:journalctl@ceph.mds.b.smithi053.stdout:Apr 17 08:05:26 smithi053 bash[71620]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-b 2024-04-17T08:05:27.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.b.service' 2024-04-17T08:05:27.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:27.318 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-17T08:05:27.318 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-17T08:05:27.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.e 2024-04-17T08:05:27.721 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:27 smithi053 systemd[1]: Stopping Ceph mds.e for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:27.721 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-e[66766]: 2024-04-17T08:05:27.636+0000 7fd11bc59700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:27.721 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:27 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-e[66766]: 2024-04-17T08:05:27.636+0000 7fd11bc59700 -1 mds.e *** got signal Terminated *** 2024-04-17T08:05:31.919 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:31 smithi053 podman[71753]: 2024-04-17 08:05:31.643506002 +0000 UTC m=+4.099971504 container died ed7c78c5c86c2ee6ba6f2c20ea50630a7e70cdeff9ac0a99dcdd05b4b629d484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T08:05:32.220 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:32 smithi053 podman[71753]: 2024-04-17 08:05:32.086500015 +0000 UTC m=+4.542965514 container cleanup ed7c78c5c86c2ee6ba6f2c20ea50630a7e70cdeff9ac0a99dcdd05b4b629d484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-17T08:05:32.221 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:32 smithi053 bash[71753]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-e 2024-04-17T08:05:32.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.e.service' 2024-04-17T08:05:32.543 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:32 smithi053 podman[71768]: 2024-04-17 08:05:32.279601296 +0000 UTC m=+0.627215727 container remove ed7c78c5c86c2ee6ba6f2c20ea50630a7e70cdeff9ac0a99dcdd05b4b629d484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:05:32.543 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:32 smithi053 systemd[1]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-17T08:05:32.543 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:32 smithi053 systemd[1]: Stopped Ceph mds.e for 67cd6c06-fc8e-11ee-bc91-c7b262605968. 2024-04-17T08:05:32.543 INFO:journalctl@ceph.mds.e.smithi053.stdout:Apr 17 08:05:32 smithi053 systemd[1]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.e.service: Consumed 1.033s CPU time. 2024-04-17T08:05:32.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:32.555 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-17T08:05:32.556 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-17T08:05:32.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.h 2024-04-17T08:05:32.971 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:05:32 smithi053 systemd[1]: Stopping Ceph mds.h for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:33.471 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:05:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h[67073]: 2024-04-17T08:05:33.009+0000 7fb72ac33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:33.472 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:05:33 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h[67073]: 2024-04-17T08:05:33.009+0000 7fb72ac33700 -1 mds.h *** got signal Terminated *** 2024-04-17T08:05:35.122 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:05:34 smithi053 podman[71875]: 2024-04-17 08:05:34.876395581 +0000 UTC m=+2.087922824 container died cc51d3d83da7b612b9c11f1280f5b4b9eb2181bf3b1980bb36413b2571a329b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, ceph=True) 2024-04-17T08:05:35.357 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:05:35 smithi053 podman[71875]: 2024-04-17 08:05:35.215372778 +0000 UTC m=+2.426900014 container cleanup cc51d3d83da7b612b9c11f1280f5b4b9eb2181bf3b1980bb36413b2571a329b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-17T08:05:35.358 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:05:35 smithi053 bash[71875]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h 2024-04-17T08:05:35.358 INFO:journalctl@ceph.mds.h.smithi053.stdout:Apr 17 08:05:35 smithi053 podman[71889]: 2024-04-17 08:05:35.358080574 +0000 UTC m=+0.476138601 container remove cc51d3d83da7b612b9c11f1280f5b4b9eb2181bf3b1980bb36413b2571a329b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-17T08:05:35.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.h.service' 2024-04-17T08:05:35.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:35.609 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-17T08:05:35.609 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-17T08:05:35.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.k 2024-04-17T08:05:35.970 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 08:05:35 smithi053 systemd[1]: Stopping Ceph mds.k for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:36.471 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 08:05:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k[66442]: 2024-04-17T08:05:36.052+0000 7f0813417700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:36.471 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 08:05:36 smithi053 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k[66442]: 2024-04-17T08:05:36.052+0000 7f0813417700 -1 mds.k *** got signal Terminated *** 2024-04-17T08:05:37.652 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 08:05:37 smithi053 podman[71996]: 2024-04-17 08:05:37.39684613 +0000 UTC m=+1.560442905 container died 236db599450973b48fb2c439d2038a568dacef853d03dcd7dcc4fc9ff13c673f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T08:05:37.970 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 08:05:37 smithi053 podman[71996]: 2024-04-17 08:05:37.752763332 +0000 UTC m=+1.916360098 container cleanup 236db599450973b48fb2c439d2038a568dacef853d03dcd7dcc4fc9ff13c673f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T08:05:37.971 INFO:journalctl@ceph.mds.k.smithi053.stdout:Apr 17 08:05:37 smithi053 bash[71996]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-k 2024-04-17T08:05:38.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.k.service' 2024-04-17T08:05:38.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:38.274 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-17T08:05:38.274 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-17T08:05:38.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.c 2024-04-17T08:05:38.575 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:05:38 smithi062 systemd[1]: Stopping Ceph mds.c for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:39.052 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:05:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c[67682]: 2024-04-17T08:05:38.576+0000 7f8bea02b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:39.052 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:05:38 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c[67682]: 2024-04-17T08:05:38.576+0000 7f8bea02b700 -1 mds.c *** got signal Terminated *** 2024-04-17T08:05:43.009 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:05:42 smithi062 podman[72441]: 2024-04-17 08:05:42.588180562 +0000 UTC m=+4.111244201 container died 0d86076c3e17edd1129697053a9be14a3d80bc7ec93f36a2686c65a5091d2360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-17T08:05:43.301 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:05:43 smithi062 podman[72441]: 2024-04-17 08:05:43.093086515 +0000 UTC m=+4.616150141 container cleanup 0d86076c3e17edd1129697053a9be14a3d80bc7ec93f36a2686c65a5091d2360 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-17T08:05:43.301 INFO:journalctl@ceph.mds.c.smithi062.stdout:Apr 17 08:05:43 smithi062 bash[72441]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-c 2024-04-17T08:05:43.455 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.c.service' 2024-04-17T08:05:43.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:43.497 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-17T08:05:43.497 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-17T08:05:43.497 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.f 2024-04-17T08:05:43.829 INFO:journalctl@ceph.mds.f.smithi062.stdout:Apr 17 08:05:43 smithi062 systemd[1]: Stopping Ceph mds.f for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:44.302 INFO:journalctl@ceph.mds.f.smithi062.stdout:Apr 17 08:05:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-f[67374]: 2024-04-17T08:05:43.830+0000 7f9efe0f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:44.302 INFO:journalctl@ceph.mds.f.smithi062.stdout:Apr 17 08:05:43 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-f[67374]: 2024-04-17T08:05:43.830+0000 7f9efe0f2700 -1 mds.f *** got signal Terminated *** 2024-04-17T08:05:47.129 INFO:journalctl@ceph.mds.f.smithi062.stdout:Apr 17 08:05:46 smithi062 podman[72563]: 2024-04-17 08:05:46.880074609 +0000 UTC m=+3.147896655 container died ab2a5170b6c852487cdd9cbd12348c79bf58623ffaa6c0e2965ff3b285f69165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-17T08:05:47.552 INFO:journalctl@ceph.mds.f.smithi062.stdout:Apr 17 08:05:47 smithi062 podman[72563]: 2024-04-17 08:05:47.205289227 +0000 UTC m=+3.473111262 container cleanup ab2a5170b6c852487cdd9cbd12348c79bf58623ffaa6c0e2965ff3b285f69165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-17T08:05:47.552 INFO:journalctl@ceph.mds.f.smithi062.stdout:Apr 17 08:05:47 smithi062 bash[72563]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-f 2024-04-17T08:05:47.850 INFO:journalctl@ceph.mds.f.smithi062.stdout:Apr 17 08:05:47 smithi062 podman[72575]: 2024-04-17 08:05:47.581829069 +0000 UTC m=+0.702471637 container remove ab2a5170b6c852487cdd9cbd12348c79bf58623ffaa6c0e2965ff3b285f69165 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-17T08:05:47.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.f.service' 2024-04-17T08:05:47.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:47.900 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-17T08:05:47.900 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-17T08:05:47.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.i 2024-04-17T08:05:48.224 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:05:47 smithi062 systemd[1]: Stopping Ceph mds.i for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:48.552 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:05:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i[67994]: 2024-04-17T08:05:48.225+0000 7f0a21775700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:48.552 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:05:48 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i[67994]: 2024-04-17T08:05:48.225+0000 7f0a21775700 -1 mds.i *** got signal Terminated *** 2024-04-17T08:05:51.174 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:05:50 smithi062 podman[72682]: 2024-04-17 08:05:50.880111436 +0000 UTC m=+2.760983685 container died d3bbd843e1eba160f2c6dae24955265d8452e84677f3886647e94e48d1bdd3db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-17T08:05:51.174 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:05:51 smithi062 podman[72682]: 2024-04-17 08:05:51.175752281 +0000 UTC m=+3.056624512 container cleanup d3bbd843e1eba160f2c6dae24955265d8452e84677f3886647e94e48d1bdd3db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-17T08:05:51.470 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:05:51 smithi062 bash[72682]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i 2024-04-17T08:05:51.470 INFO:journalctl@ceph.mds.i.smithi062.stdout:Apr 17 08:05:51 smithi062 podman[72694]: 2024-04-17 08:05:51.385709128 +0000 UTC m=+0.505902621 container remove d3bbd843e1eba160f2c6dae24955265d8452e84677f3886647e94e48d1bdd3db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-17T08:05:51.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.i.service' 2024-04-17T08:05:51.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:51.524 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-17T08:05:51.525 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-17T08:05:51.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.l 2024-04-17T08:05:51.802 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 08:05:51 smithi062 systemd[1]: Stopping Ceph mds.l for 67cd6c06-fc8e-11ee-bc91-c7b262605968... 2024-04-17T08:05:52.302 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 08:05:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l[67057]: 2024-04-17T08:05:51.854+0000 7ff232e03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T08:05:52.302 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 08:05:51 smithi062 ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l[67057]: 2024-04-17T08:05:51.854+0000 7ff232e03700 -1 mds.l *** got signal Terminated *** 2024-04-17T08:05:54.879 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 08:05:54 smithi062 podman[72804]: 2024-04-17 08:05:54.615845347 +0000 UTC m=+2.866478075 container died a56f0800ff739480d0787ada4535d865757bd6af5ac617b7071a9dc084d1bb91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-17T08:05:55.302 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 08:05:54 smithi062 podman[72804]: 2024-04-17 08:05:54.980586506 +0000 UTC m=+3.231219236 container cleanup a56f0800ff739480d0787ada4535d865757bd6af5ac617b7071a9dc084d1bb91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T08:05:55.302 INFO:journalctl@ceph.mds.l.smithi062.stdout:Apr 17 08:05:54 smithi062 bash[72804]: ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968-mds-l 2024-04-17T08:05:55.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-67cd6c06-fc8e-11ee-bc91-c7b262605968@mds.l.service' 2024-04-17T08:05:55.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T08:05:55.606 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-17T08:05:55.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T08:05:55.950 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T08:06:01.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T08:06:01.295 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T08:06:03.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T08:06:04.139 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T08:06:06.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T08:06:06.816 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T08:06:06.856 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T08:06:06.889 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-17T08:06:06.891 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi046/crash 2024-04-17T08:06:06.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash -- . 2024-04-17T08:06:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T08:06:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-04-17T08:06:07.023 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi053/crash 2024-04-17T08:06:07.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash -- . 2024-04-17T08:06:07.066 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T08:06:07.066 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-04-17T08:06:07.068 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi062/crash 2024-04-17T08:06:07.069 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash -- . 2024-04-17T08:06:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T08:06:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-04-17T08:06:07.112 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-17T08:06:07.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-17T08:06:07.232 INFO:tasks.cephadm:Compressing logs... 2024-04-17T08:06:07.232 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T08:06:07.237 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T08:06:07.239 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T08:06:07.272 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T08:06:07.272 INFO:teuthology.orchestra.run.smithi062.stderr:: No such file or directory 2024-04-17T08:06:07.272 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.l.log 2024-04-17T08:06:07.272 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T08:06:07.272 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.i.log 2024-04-17T08:06:07.272 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.11.log 2024-04-17T08:06:07.273 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.z.log 2024-04-17T08:06:07.273 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.8.log 2024-04-17T08:06:07.273 INFO:teuthology.orchestra.run.smithi062.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T08:06:07.273 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.5.log 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.z.log: /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.f.log 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.f.log: 90.2% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log: 94.7% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.l.log.gz 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr: 82.3% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.2.log 2024-04-17T08:06:07.285 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.c.log 2024-04-17T08:06:07.286 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.2.log: 86.7% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T08:06:07.286 INFO:teuthology.orchestra.run.smithi062.stderr: 92.5% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.z.log.gz 2024-04-17T08:06:07.286 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T08:06:07.286 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.c.log 2024-04-17T08:06:07.289 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.10.log 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.b.log 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.10.log: /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.4.log 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.1.log 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T08:06:07.290 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.k.log 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.h.log 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.7.log 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.e.log 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.7.log: 90.2% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.y.log 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.e.log: 94.7% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.k.log.gz 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.y.log: 94.6% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.e.log.gz 2024-04-17T08:06:07.291 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log 2024-04-17T08:06:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log: 82.4% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T08:06:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T08:06:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log: 86.6% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T08:06:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.b.log 2024-04-17T08:06:07.292 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log: 92.4% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.y.log.gz 2024-04-17T08:06:07.301 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T08:06:07.306 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log: /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T08:06:07.306 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-17T08:06:07.307 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T08:06:07.307 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-17T08:06:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.6.log 2024-04-17T08:06:07.309 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.a.log 2024-04-17T08:06:07.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T08:06:07.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.a.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-17T08:06:07.312 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.g.log 2024-04-17T08:06:07.313 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.j.log 2024-04-17T08:06:07.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.3.log 2024-04-17T08:06:07.317 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.x.log 2024-04-17T08:06:07.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.3.log: 90.0% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T08:06:07.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.x.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T08:06:07.319 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.0.log 2024-04-17T08:06:07.321 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T08:06:07.321 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.j.log.gz 2024-04-17T08:06:07.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log 2024-04-17T08:06:07.324 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T08:06:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T08:06:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.9.log 2024-04-17T08:06:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.d.log 2024-04-17T08:06:07.330 INFO:teuthology.orchestra.run.smithi046.stderr: 86.7% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T08:06:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.d.log: /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.a.log 2024-04-17T08:06:07.346 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.a.log: 90.9% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T08:06:07.348 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.d.log.gz 2024-04-17T08:06:07.355 INFO:teuthology.orchestra.run.smithi046.stderr: 94.5% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.a.log.gz 2024-04-17T08:06:07.580 INFO:teuthology.orchestra.run.smithi046.stderr: 89.9% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mgr.x.log.gz 2024-04-17T08:06:07.626 INFO:teuthology.orchestra.run.smithi062.stderr: 94.7% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.c.log.gz 2024-04-17T08:06:07.670 INFO:teuthology.orchestra.run.smithi062.stderr: 94.2% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.f.log.gz 2024-04-17T08:06:07.671 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.b.log.gz 2024-04-17T08:06:07.687 INFO:teuthology.orchestra.run.smithi062.stderr: 91.7% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.c.log.gz 2024-04-17T08:06:07.996 INFO:teuthology.orchestra.run.smithi046.stderr: 90.4% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mon.a.log.gz 2024-04-17T08:06:08.277 INFO:teuthology.orchestra.run.smithi062.stderr: 95.2% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.i.log.gz 2024-04-17T08:06:08.713 INFO:teuthology.orchestra.run.smithi053.stderr: 92.3% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.10.log.gz 2024-04-17T08:06:08.968 INFO:teuthology.orchestra.run.smithi053.stderr: 92.1% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.1.log.gz 2024-04-17T08:06:09.483 INFO:teuthology.orchestra.run.smithi062.stderr: 92.8% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.8.log.gz 2024-04-17T08:06:09.503 INFO:teuthology.orchestra.run.smithi046.stderr: 91.9% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.6.log.gz 2024-04-17T08:06:09.757 INFO:teuthology.orchestra.run.smithi053.stderr: 92.0% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.7.log.gz 2024-04-17T08:06:09.794 INFO:teuthology.orchestra.run.smithi062.stderr: 92.5% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.11.log.gz 2024-04-17T08:06:09.970 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.0.log.gz 2024-04-17T08:06:09.970 INFO:teuthology.orchestra.run.smithi046.stderr: 94.1% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.g.log.gz 2024-04-17T08:06:10.050 INFO:teuthology.orchestra.run.smithi062.stderr: 92.4% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.2.log.gz 2024-04-17T08:06:10.181 INFO:teuthology.orchestra.run.smithi053.stderr: 94.2% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.h.log.gz 2024-04-17T08:06:10.205 INFO:teuthology.orchestra.run.smithi046.stderr: 92.4% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.3.log.gz 2024-04-17T08:06:10.399 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.9.log.gz 2024-04-17T08:06:10.403 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-17T08:06:10.403 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m3.134s 2024-04-17T08:06:10.403 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m13.493s 2024-04-17T08:06:10.404 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m1.060s 2024-04-17T08:06:10.832 INFO:teuthology.orchestra.run.smithi062.stderr: 92.2% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.5.log.gz 2024-04-17T08:06:10.834 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-17T08:06:10.834 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m3.580s 2024-04-17T08:06:10.834 INFO:teuthology.orchestra.run.smithi062.stderr:user 0m12.846s 2024-04-17T08:06:10.834 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m0.411s 2024-04-17T08:06:10.886 INFO:teuthology.orchestra.run.smithi053.stderr: 92.4% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-osd.4.log.gz 2024-04-17T08:06:19.488 INFO:teuthology.orchestra.run.smithi053.stderr: 93.5% -- replaced with /var/log/ceph/67cd6c06-fc8e-11ee-bc91-c7b262605968/ceph-mds.b.log.gz 2024-04-17T08:06:19.490 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-04-17T08:06:19.490 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m12.236s 2024-04-17T08:06:19.490 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m24.088s 2024-04-17T08:06:19.491 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.673s 2024-04-17T08:06:19.491 INFO:tasks.cephadm:Archiving logs... 2024-04-17T08:06:19.491 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi046/log 2024-04-17T08:06:19.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T08:06:22.232 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi053/log 2024-04-17T08:06:22.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T08:06:25.586 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi062/log 2024-04-17T08:06:25.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T08:06:27.451 INFO:tasks.cephadm:Removing cluster... 2024-04-17T08:06:27.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 --force 2024-04-17T08:06:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T08:06:28.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 --force 2024-04-17T08:06:28.641 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T08:06:28.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 67cd6c06-fc8e-11ee-bc91-c7b262605968 --force 2024-04-17T08:06:29.135 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 67cd6c06-fc8e-11ee-bc91-c7b262605968 2024-04-17T08:06:29.420 INFO:tasks.cephadm:Removing cephadm ... 2024-04-17T08:06:29.421 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T08:06:29.484 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T08:06:29.500 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T08:06:29.516 INFO:tasks.cephadm:Teardown complete 2024-04-17T08:06:29.517 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-17T08:06:29.527 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-17T08:06:29.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T08:06:29.536 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T08:06:29.543 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T08:06:29.703 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-17T08:06:29.716 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-17T08:06:29.716 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T08:06:29.725 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T08:06:29.727 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T08:06:29.740 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-04-17T08:06:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-17T08:06:29.745 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T08:06:29.745 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-04-17T08:06:29.745 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +825us[ +826us] +/- 54ms 2024-04-17T08:06:29.745 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -951us[ -950us] +/- 81ms 2024-04-17T08:06:29.745 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -428us[ -427us] +/- 91ms 2024-04-17T08:06:29.745 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T08:06:29.747 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T08:06:29.747 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-17T08:06:29.747 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +764us[ +772us] +/- 53ms 2024-04-17T08:06:29.747 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -982us[ -982us] +/- 81ms 2024-04-17T08:06:29.747 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 -476us[ -468us] +/- 91ms 2024-04-17T08:06:29.747 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T08:06:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-04-17T08:06:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T08:06:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-04-17T08:06:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 7 377 65 +799us[ +799us] +/- 53ms 2024-04-17T08:06:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 64 -961us[ -961us] +/- 81ms 2024-04-17T08:06:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -497us[ -497us] +/- 91ms 2024-04-17T08:06:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T08:06:29.788 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-17T08:06:29.799 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-17T08:06:29.800 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-17T08:06:29.809 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T08:06:30.062 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T08:06:30.145 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T08:06:30.225 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T08:06:30.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T08:06:30.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T08:06:30.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T08:06:30.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T08:06:30.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T08:06:30.396 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-17T08:06:30.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-17T08:06:30.419 INFO:teuthology.task.internal:Duration was 1823.719984 seconds 2024-04-17T08:06:30.420 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-17T08:06:30.430 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-17T08:06:30.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T08:06:30.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T08:06:30.440 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T08:06:30.491 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T08:06:30.492 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T08:06:30.576 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T08:06:30.833 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-17T08:06:30.834 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-04-17T08:06:30.834 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T08:06:30.905 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-04-17T08:06:30.906 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T08:06:30.940 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-04-17T08:06:30.941 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T08:06:30.965 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-17T08:06:30.965 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T08:06:30.971 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T08:06:30.983 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T08:06:31.064 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-17T08:06:31.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T08:06:31.070 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T08:06:31.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T08:06:31.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-17T08:06:31.609 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-17T08:06:31.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T08:06:31.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T08:06:31.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T08:06:31.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-17T08:06:31.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T08:06:31.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T08:06:31.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T08:06:31.815 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-04-17T08:06:31.836 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-04-17T08:06:31.884 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-04-17T08:06:31.943 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T08:06:31.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T08:06:31.993 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T08:06:32.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T08:06:32.007 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T08:06:32.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T08:06:32.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-17T08:06:32.033 INFO:teuthology.task.internal:Transferring archived files... 2024-04-17T08:06:32.034 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi046 2024-04-17T08:06:32.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T08:06:32.155 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi053 2024-04-17T08:06:32.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T08:06:32.197 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912/remote/smithi062 2024-04-17T08:06:32.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T08:06:32.241 INFO:teuthology.task.internal:Removing archive directory... 2024-04-17T08:06:32.241 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T08:06:32.249 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T08:06:32.251 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T08:06:32.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-17T08:06:32.315 INFO:teuthology.task.internal:Not uploading archives. 2024-04-17T08:06:32.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-17T08:06:32.326 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-17T08:06:32.327 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T08:06:32.335 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T08:06:32.337 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T08:06:32.349 INFO:teuthology.orchestra.run.smithi053.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 08:06 /home/ubuntu/cephtest 2024-04-17T08:06:32.351 INFO:teuthology.orchestra.run.smithi062.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 08:06 /home/ubuntu/cephtest 2024-04-17T08:06:32.387 INFO:teuthology.orchestra.run.smithi046.stdout: 393338 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 08:06 /home/ubuntu/cephtest 2024-04-17T08:06:32.391 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-17T08:06:32.402 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-17T08:06:32.426 INFO:teuthology.nuke:Checking targets against current locks 2024-04-17T08:06:32.452 DEBUG:teuthology.nuke:shortname: smithi046 2024-04-17T08:06:32.452 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T08:06:32.474 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 07:26:14.683903', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI8YCbSQKJnZDWMiRzOO/nNlJCnrRrKWAdZnsZ32+4q5PTZL7pN1fyrKpFrN5JA8nfZTtBGAOvydLOupfZsAYc4='} 2024-04-17T08:06:32.478 DEBUG:teuthology.nuke:shortname: smithi053 2024-04-17T08:06:32.478 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T08:06:32.501 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 07:26:14.682253', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWzDvJkyYvkxzcFgX+K/t5Qfyb7gpPrXpt0dSdzfRIF/g6onXa7y0plhTMuaHKUQ9mGzgXdtUprrzPnWvn0Cxg='} 2024-04-17T08:06:32.504 DEBUG:teuthology.nuke:shortname: smithi062 2024-04-17T08:06:32.504 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T08:06:32.529 INFO:teuthology.orchestra.console.smithi046:Power off 2024-04-17T08:06:32.529 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T08:06:32.543 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659912', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 07:26:14.680369', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHUEHVICN3K3rqyRYx+RBRTycjx+SxXCcH+d9/LUJM9/Ppo+I0EmyvFZIMRwVwsh+OFC0XT8TKvE5uIjU+ko6fk='} 2024-04-17T08:06:32.554 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2024-04-17T08:06:32.554 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:32.570 INFO:teuthology.orchestra.console.smithi053:Power off 2024-04-17T08:06:32.571 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T08:06:32.584 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-04-17T08:06:32.596 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-04-17T08:06:32.596 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:32.610 INFO:teuthology.orchestra.console.smithi062:Power off 2024-04-17T08:06:32.610 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T08:06:32.619 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-17T08:06:32.633 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-04-17T08:06:32.634 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:32.656 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-17T08:06:36.586 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:36.621 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:36.657 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:36.697 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-04-17T08:06:36.731 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-17T08:06:36.766 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-17T08:06:40.702 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:40.731 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:40.768 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:40.813 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2024-04-17T08:06:40.841 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-17T08:06:40.877 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-04-17T08:06:40.915 INFO:teuthology.orchestra.console.smithi046:Power off completed 2024-04-17T08:06:40.978 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-04-17T08:06:44.844 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:06:44.955 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-04-17T08:06:45.055 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-04-17T08:06:45.163 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} duration: 1823.7199835777283 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi046 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=09e02bd6081744cdb6e54d9be0db1ba4 status: fail success: false 2024-04-17T08:06:45.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T08:06:45.249 INFO:teuthology.run:FAIL